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ABSTRACT 


Organizations  are  becoming  increasingly  aware  that  learning  can  be  a  source  of 
competitive  advantage.  The  United  States  Navy  is  not  immune  to  this  realization  and  has 
established  the  Navy  Lessons  Learned  System  (NLLS)  as  the  singular  Navy  program  for 
the  collection,  validation,  and  distribution  of  unit  feedback.  NLLS  provides  naval 
personnel  a  means  to  share  observations,  document  deficiencies,  convey  solutions,  and 
innovate  tactics,  techniques,  or  procedures  (TTP).  The  purpose  of  this  thesis  is  to 
examine  the  various  factors  that  influence  organizational  learning,  such  as  structure, 
environment,  and  culture,  and  to  examine  how  Information  Technology  can  be  used  to 
support  or  enhance  organizational  learning  in  the  Navy.  The  research  concludes  that 
NLLS  has  improved  organizational  learning  but  has  not  attained  as  widespread  use  as  is 
possible.  Recommendations  are  provided  to  improve  the  program  as  well  as  increase 
NLLS  exposure  to  the  fleet  and  to  the  potential  users  of  the  system. 
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I.  INTRODUCTION 


Chris  Argyris  and  Donald  A.  Schon,  in  their  classic  book  on  organizational 
learning,  define  it  as  the  process  of  “detection  and  correction  of  errors.”  In  their  view, 
organizations  learn  through  the  individuals  who  act  as  agents  for  them:  “The  individuals’ 
learning  activities  in  turn,  are  facilitated  or  inhibited  by  an  ecological  system  of  factors 
that  may  be  called  an  organization  learning  system.”  (Argyris  and  Schon,  1978)  George 
Huber  expands  on  this,  stating,  “An  organization  learns  when,  through  its  processing  of 
information,  the  range  or  likelihood  of  its  potential  behaviors  is  changed.”  (Huber,  1991) 
Offering  us  a  procedural  definition  are  other  noted  authors,  Raanan  Lipshitz  and 
Micha  Popper  who  define  a  learning  organization  as: 

A  learning  organization  is  an  organization  that  institutes  organization¬ 
learning  mechanisms  and  operates  them  regularly.  Organization  Learning 
Mechanisms  are  defined  as  institutionalized  structural  and  procedural 
arrangements  that  allow  organizations  to  systematically  collect,  analyze, 
store,  disseminate,  and  use  information  that  is  relevant  to  the  effectiveness 
of  the  organization.  (Lipshitz  and  Popper,  1996) 

Organizations  as  diverse  as  Motorola,  BP  Amoco,  and  the  United  States  Army  are 
becoming  increasingly  aware  of  the  concept  of  organizational  learning  in  order  to  acquire 
a  competitive  advantage,  increase  their  capacity  to  innovate,  or  improve  effectiveness. 

With  this  need  in  mind,  every  day  around  the  world  naval  units  are  involved  in 
either  real  world  exercises  or  training  missions  that  mirror  the  real  world.  Traditionally, 
emphasis  is  placed  on  the  preparation-through-execution  phase.  However,  if  we  do  not 
review,  analyze,  and  disseminate  the  exercises  and  missions  we  have  just  concluded,  we 
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lose  potentially  the  most  important  aspect:  the  lessons  learned,  and  with  them  our  ability 
as  an  organization  to  learn.  While  this  may  seem  a  simple  enough  concept,  actualizing  it 
requires  two  major  elements.  First,  a  way  to  collect  and  record  these  valuable 
contributions,  and  second,  a  way  to  share  the  information  throughout  the  organization. 
Without  both  we  are  destined  to  repeat  past  mistakes. 

A.  PURPOSE 

This  research  evaluates  the  impact  that  the  implementation  of  an  Information 
Technology  infrastructure  has  had  on  the  potential  for  organizational  learning  through  the 
Navy  Lessons  Learned  System  (NLLS).  The  purpose  of  this  study  is  to  explore  and 
discuss  the  influence  that  technology,  especially  information  technology,  has  had  on  the 
organizational  learning  process  in  the  Navy. 

The  primary  research  objective  is  to  explore  the  answer  to  the  following  question: 
Can  the  use  of  Information  Technology  in  the  Navy’s  Lessons  Learned  System  be  used  to 
improve  organizational  learning?  Secondary  questions  include:  How  effective  is  the 
Navy  Lessons  Learned  System  as  a  tool  for  promoting  organizational  learning?  Is 
Information  Technology  being  used  effectively  to  analyze  collected  information  to 
identify  a  recurring  weakness  in  tactics,  techniques,  or  procedures?  And  lastly,  to  what 
degree  has  Information  Technology  improved  the  process  or  availability  of  disseminating 
relevant  information  to  potential  users? 

This  research  targets  the  information  systems  at  the  Center  for  Navy  Lessons 
Learned.  A  secondary  purpose  of  this  thesis  is  to  provide  the  Naval  Warfare  and 


2 


Development  Center  with  recommendations  for  improving  and  enhancing  the  existing 
organizational  learning  process  as  evaluated  against  that  of  other  learning  organizations. 

B.  BACKGROUND 

Today,  in  an  era  of  declining  defense  dollars,  the  Department  of  Defense  and  the 
United  States  Navy  face  continually  increasing  pressures  to  reduce  spending.  They  are 
responding  by  reducing  the  numbers  of  ships  and  aircraft,  downsizing  military  personnel, 
and  decreasing  training  opportunities  for  those  who  stay.  To  surmount  the  budget 
challenges,  the  military  must  reap  the  maximum  benefit  from  the  opportunities  still 
available. 

The  overriding  objective  of  the  Navy’s  Lessons  Learned  program  is  to  provide  a 
fleet-wide  benefit  from  operational  exercises  and  experiences.  To  be  successful  in  this 
endeavor,  the  program  must  create  a  coordinated  process  for  capturing,  securing, 
retrieving,  and  distributing  the  organization’s  knowledge. 

A  common  phrase  heard  in  the  business  world  is,  “If  only  we  knew  what  we 
know,  we  would  be  twice  as  profitable.”  But  why  is  there  a  need  for  the  Navy  to 
encourage  knowledge  sharing?  We  do  not  contend  in  the  same  competitive  marketplaces 
as  corporate  America;  nor  is  the  United  States  currently  at  war  with  anyone.  However, 
like  all  government  organizations,  today’s  military  is  competing  for  scarce  resources  and 
must  take  full  advantage  of  the  opportunities  presented.  Consequently,  we  have  seen 
lessons  learned  in  training  exercises  that  often  must  be  relearned  in  conflict,  sometimes 
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with  fatal  results.  A  GAO  study  published  in  1994  points  out  one  such  example  that 
occurred  during  the  recent  military  experience  in  the  Persian  Gulf: 

“During  Operation  Desert  Shield  in  1990-1991,  Jive  Army  protective 
bunker  collapses  caused  three  fatalities  and  three  other  injuries.  The 
cause  sited  in  a  subsequent  lessons  learned  report  was  that  ...locally 
available  materials  were  used  to  construct  bunkers  deviating  from 
standard  design.  ”  (GAO,  1994) 

In  1988  the  Center  for  Army  Lessons  Learned  (CALL)  had  reports  that  units  at  the 
combat  training  centers  had  difficulty  in  obtaining  support  materials  for  their  protective 
bunkers,  which  caused  them  to  use  locally  available  materials.  The  results  were  “poorly 
constructed  bunkers  made  of  non-standard  design  and  materials.”  This  prompted  a 
lessons  learned  report  that  was  disseminated  by  CALL  in  1988.  Because  of  “inadequate 
follow-up  and  corrective  action,”  a  similar  lessons  learned  report  had  to  be  generated  in 
1991,  this  time  after  a  lesson  with  far  more  severe  consequences.  (GAO,  1994) 

C.  SCOPE  AND  METHODOLOGY 

The  scope  of  this  study  focuses  on  the  use  of  the  Navy  Lessons  Learned  System 
(NLLS)  as  a  structured  method  for  organizational  learning.  The  primary  concentration  of 
the  study  is  the  use  of  organizational  learning  constructs  and  models  as  a  framework  for 
comparing  procedures  in  place  at  the  Naval  Warfare  and  Doctrine  Command  and  at  least 
one  other  military  organization.  Current  IT  policies  are  examined  to  ascertain  if 
recommendations  or  conclusions  can  be  made  to  aid  in  improving  the  existing  IT 
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infrastructure  at  NLLS.  The  goal  is  to  increase  NLLS  ability  to  provide  the  best 
relationship  between  the  information  system  organization  and  the  customers  they  support. 

D.  RESEARCH  METHODOLOGY 

The  research  techniques  used  for  this  thesis  comprise  the  following  steps: 

1 .  Conduct  a  thorough  literature  search  of  the  NLLS  mission,  purpose,  history, 
and  organizational  structure  from  documents  provided  by  the  Navy  Warfare  Development 
Command. 

2.  Gather  technical  information  about  the  IT  infrastructure  and  collection  process 
by  conducting  telephone  interviews  with  relevant  members  of  the  organization. 

3.  Conduct  a  literature  search  of  Naval  Instructions,  Naval  Programs,  professional 
journals,  books,  magazine  articles,  and  other  library  informational  resources  about  the 
Navy  Lessons  Learned  System. 

4.  Conduct  a  literature  review  of  different  learning  organizational  design  models 
to  explore  and  discuss  the  uses  of  their  key  components  for  this  application. 

5.  Conduct  a  literature  review  of  the  Center  for  Army  Lessons  Learned  program 
to  explore  and  discuss  possible  benefits  or  drawbacks. 

6.  Explore  the  ability  of  NLLS  to  deliver  value  to  the  user  by  employing 
emerging  Information  Technologies  to  prevent  recurring  weakness  in  training,  techniques, 
or  procedures. 

7.  Generate  a  sample-based  assessment  of  NLLS ’s  impact  on  the  organization, 
using  an  appropriate  survey. 
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E.  ORGANIZATION  OF  STUDY 


The  thesis  is  organized  as  follows. 

Chapter  I  Introduction:  This  chapter  provides  the  purpose,  background,  and 
research  methodology  of  the  thesis. 

Chapter  II  Naval  Warfare  and  Development  Command:  This  chapter  provides 
background  information  on  the  organization  of  the  Navy  Warfare  Development 
Command,  history  of  the  Navy  Lesson  Learned  System,  methodology  of  collection  at  the 
Fleet  Management  sites,  and  validation  of  inputs  through  interaction  of  the  Subject 
Matter  Experts. 

Chapter  III  Organizational  Learning  Models:  This  chapter  discusses  individual 
learning  verses  organizational  learning  and  presents  the  reader  with  several  types  of 
organizational  learning  models.  The  effect  that  structure,  environment,  and  culture  can 
have  on  learning  is  also  discussed. 

Chapter  IV  Use  of  Technology  in  a  Learning  Organization:  This  chapter  will  look 
at  how  to  build  a  Learning  Organization  and  examine  the  use  of  technology  to  improve 
Organizational  Learning. 

Chapter  V  Information  Technology  in  Other  Organizations:  This  chapter  will 
provide  a  case  study  of  organizational  learning  as  it  occurs  in  another  military 
organization  for  comparison  purposes. 

Chapter  VI  Findings:  The  chapter  will  present  the  survey  questions  with  a 
breakdown  of  the  responses  received. 
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Chapter  VII  Analysis:  This  chapter  will  conduct  an  analysis  of  the  results  of  the 


survey. 

Chapter  VIII  Conclusions  and  Recommendations.  This  chapter  will  provide 
conclusions  and  recommendations  to  improve  the  effectiveness  of  NLLS. 
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II.  NAVAL  WARFARE  DEVELOPMENT  COMMAND 


A.  INTRODUCTION 


The  Naval  Warfare  Development  Command  was  officially  established  on  10 
August  1998  in  Newport,  Rhode  Island.  Its  purpose  is  to  lead  warfare  innovation  and 
concept  development,  to  design  plan  and  coordinate  the  Navy’s  Fleet  Battle  Experiment 
program,  and  to  represent  the  Navy  in  Joint  Experimentation.  NWDC  charter  is  to 
synchronize,  approve,  and  disseminate  Navy  doctrine. 

The  Navy  Warfare  Development  Command  is  composed  of  four  departments: 
Concepts  Development,  Maritime  Battle  Center,  Doctrine,  and  Operations.  (NWDC 
Public  Affairs,  2000) 


Naval  Warfare 
Development 
Command 


Doctrine 


:  At  .  ;  ; 


Figure  1 .  NWDC  Organizational  Chart 


1.  Concepts 

The  Concepts  Department’s  mission  is  to  develop  Navy  warfighting  concepts  and 
identify  required  capabilities.  These  concepts  are  then  tested  and  refined  through  Fleet 
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Battle  Experiments,  war  games,  modeling  and  simulations,  and  other  processes.  The 
results  are  expected  to  form  the  basis  to  develop  and  revise  Navy  Doctrine. 

By  pooling  ideas  from  many  sources  and  taking  the  most  promising  ideas. 
Concepts  can  develop  and  identify  capabilities  with  potential  improvements  to  the  way 
the  Navy  operates.  These  ideas  are  then  validated  through  a  fast-paced,  six-month 
experimentation  process  providing  for  warfighting  innovation.  (NWDC  Public  Affairs, 
2000) 


2.  Maritime  Battle  Center 

The  Maritime  Battle  Center  (MBC)  was  established  to  serve  as  the  single  point  of 
contact  for  the  Navy  Fleet  Battle  Experimentation  (FBE)  and  participation  in  Joint 
Experiments. 

The  MBC  is  responsible  for  designing  and  planning  Fleet  Battle  Experiments, 
coordinating  the  execution  of  these  experiments  in  conjunction  with  the  numbered  fleet 
operational  command  elements,  and  analyzing  and  disseminating  experiment  results.  The 
FBE  results  are  then  used  to  accelerate  the  delivery  of  innovative  warfare  capabilities  to 
the  fleet,  identify  concept  based  requirements,  and  evaluate  new  operational  capabilities. 
(NWDC  Public  Affairs,  2000) 

3.  Doctrine 

The  Doctrine  Department  is  the  Navy’s  primary  point  of  contact  for  Naval 
Doctrine  and  Joint  and  Combined  Doctrine  Development.  As  the  Fleet  agent  for 
operational  doctrine  development,  the  NWDC  Doctrine  Department  facilitates  fleet 


10 


consensus  on  emerging  Navy  and  Joint  doctrinal  issues.  (NWDCINST  5400.  IX)  In 
addition,  the  Doctrine  Department  has  the  administrative  and  management  responsibility 
for  the  Navy  Warfare  Publication  Library  and  the  Navy  Lessons  Learned  System. 
(NWDC  Public  Affairs,  2000) 

The  primary  means  for  the  proposal,  validation,  and  revision  of  Navy  operational- 
level  doctrine  and  inputs  to  Joint  doctrine  is  through  the  use  of  a  semi-annual  Navy 
Doctrine  Working  Party  (NDWP).  The  objectives  of  the  NDWP  are  threefold: 

•  Serve  as  a  forum  for  Fleet  engagement  on  current  or  emerging  doctrinal 
issues. 

•  Consolidate  the  Fleet  input  on  topics  at  the  subsequent  Joint  Doctrine  Working 
Party. 

•  Validate  proposals  for  development/revision  of  Navy  Warfare  Publications 
(NWP). 

According  to  Captain  Robert  Nestlerode,  Doctrine  Department  Head,  proven 
operational  concepts,  validated  through  experimentation  and  collaborative  development 
with  the  fleet,  translate  into  useful  doctrine.  “The  goal  is  to  make  the  Navy’s  doctrine 
development  dynamic,  responsive,  and  interactive.”  (Nestlerode,  2000) 

4.  Operations 

The  Operations  Department  is  the  newest  department  at  NWDC,  established  in 
May  1999.  The  purpose  of  the  Operations  Department  is  to  provide  support  in  the  form  of 
technical  research,  analysis,  modeling  and  simulation,  and  "red  cell,"  or  adversary  view, 
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of  emerging  concepts.  In  addition,  the  Operations  Department  is  tasked  to  act  as  the 
NWDC  implementation  conduit  to  the  Fleet  Commanders  in  Chief,  Numbered  Fleet 
Commanders,  Naval  Systems  Command,  and  the  Integrated  Warfare  Requirements 
(IWAR)  board  in  Washington  D.C.  (NWDC  Public  Affairs,  2000) 

B.  OVERVIEW  OF  THE  NAVY  LESSONS  LEARNED  SYSTEM 

1.  Navy  Lessons  Learned  Central  Site 

a.  History 

The  Navy  Lessons  Learned  System  (NLLS)  was  formally  created  in  1991 
by  CNO  direction  as  a  result  of  a  growing  fleet  interest  in  providing  a  centrally  managed 
and  easily  accessible  lessons  learned  database.  In  1995,  a  Congressionally  ordered  GAO 
report  was  released  that  was  critical  of  all  the  services  and  their  potential  to  use  lessons 
learned  as  a  means  to  avoid  repetition  of  past  mistakes.  Consequently,  the  Chief  of  Naval 
Operations  improved  on  this  system  by  instituting  and  formalizing  guidance  in  the  form 
of  an  OPNAV  instruction.  Its  purpose  was  to  standardize  and  delineate  the  NLLS 
requirement  and  formal  feedback  process.  The  Navy  Lessons  Learned  System  that 
developed  was  patterned  after  the  Joint  Chiefs  of  Staff  (JCS)  sponsored  Joint  Universal 
Lessons  Learned  System  (JULLS).  This  system  currently  makes  up  the  formal  feedback 
process.  (NWDC  Public  Affairs,  2000) 
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b.  Purpose 

By  definition,  a  lesson  learned  is  information  that  contributes  to  the 


Navy’s  corporate  body  of  knowledge  and  should  produce  increased  process  efficiency  and 
improved  execution  of  future  operations.  Additionally,  it  should  provide  value  added  to 
existing  Navy  policy,  doctrine,  tactics,  techniques,  procedures,  training,  systems  or 
equipment.  (CINCPACFLTINST  3500.37A) 

The  objective  of  NLLS  is  to  provide  for  a  Navy- wide  standardized 
procedure  for  collection,  validation,  and  distribution  of  lessons  learned  from  fleet 
exercises  and  everyday  operations.  This  provides  Naval  activities  with  a  formally 
managed  database  of  current  information  that  has  been  validated  by  either  platform  or 
warfare  experts  in  support  of  fleet  operations.  The  lessons  learned  are  submitted  through 
the  use  of  the  Navy  Instruction  Input  Program  (NIIP)  to  the  applicable  commands 
operational  Immediate  Superior  in  Command  (ISIC).  There,  they  are  checked  for 
applicability  and  redundancy  before  being  sent  on  to  one  of  five  Fleet  Management  Sites 
for  validation  and  inclusion  in  the  database. 

c.  Structure 

The  Navy  Lessons  Learned  Central  site  is  assigned  under  the  Naval 
Warfare  Development  Command  (NWDC)  Doctrine  department  and  serves  as  the 
centralized  data  collection,  management,  and  distribution  center  for  all  NLLS  inputs.  The 
Central  Site  also  is  tasked  to  act  as  the  Program  Director  and  Administrator  for  NLLS,  as 
well  as  provide  for  the  quality  assurance  of  the  Navy  Lessons  Learned  Data  Base 
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(NLLSDB),  and  distribution  of  the  product  to  fleet  units.  Contractor  support  at  NWDC 
consolidates  Lessons  Learned,  Port  Visit  Reports,  and  After  Action  Reports,  insuring  that 
they  are  in  the  proper  format  before  releasing  the  data  as  part  of  the  NLLSDB  via  a 
Secure  Internet  Protocol  Router  Network  (SIPRNET)  Web  site  and  a  quarterly  CD-ROM 
set. 


2.  Fleet  Management  Sites 

a.  Purpose 

Fleet  Management  Sites  (FMS)  are  responsible  for  the  collection, 
processing,  and  validating  of  lessons  learned  submitted  from  their  area  of  responsibility 
(AOR).  By  supplying  the  approval  on  each  submission,  each  fleet  CINC  establishes 
control  over  the  quality  and  validity  of  the  lessons  learned  in  his  area  before  it  is 
submitted  to  the  central  site  for  inclusion  into  the  NLLDB. 

b.  Structure 

When  a  fleet  user  submits  a  lesson  learned,  it  is  sent  to  one  of  five  Fleet 
Management  Sites,  depending  on  the  originator’s  theater  of  operations.  These  sites  are 
located  with  the  numbered  fleet  commanders  and  are  each  manned  by  a  civilian 
contractor  senior  data  analyst.  It  is  this  analyst,  with  the  help  of  a  variety  of  subject  matter 
experts  (SMEs),  who  is  responsible  for  the  processing  and  validation  of  the  lessons 
learned  submitted  by  his  or  her  subordinate  commands. 
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The  Fleet  Management  Sites  are  also  responsible  for  overseeing  the 
Remedial  Action  Program  for  their  areas  and  ensuring  NLLS  training  is  conducted  aboard 
each  deploying  command.  Additionally,  they  conduct  an  annual  review  to  verify  the 
currency  and  quality  of  the  database  at  the  central  site.  (OPNAV3500.37B) 

3.  Analysis  and  Validation 

a.  Subject  Matter  Experts 

Subject  Matter  Experts  (SMEs)  are  personnel  in  major  commands,  warfare 
centers,  or  tactical  development  centers  who  are  recognized  for  having  extensive 
knowledge  and  experience  in  a  particular  warfare  area  of  operation  (OPNAVINST 
3500.37B).  SMEs  provide  the  theater  commanders  support  by  reviewing  and  validating 
Lessons  Learned  and  Remedial  Action  Program  issues  as  required.  For  example,  a  lesson 
learned  submission  on  Battle  Group  tactics  in  the  Atlantic  would  go  to  the  SMEs  located 
at  Tactical  Training  Group  Atlantic  for  review,  consideration,  and  validation.  If  it 
concerned  employment  of  a  Carrier  Air  Wing,  it  would  go  to  the  Naval  Strike  and  Air 
Warfare  Center. 

b.  Remedial  Action  Program 

A  Remedial  Action  Program  (RAP)  item  is  defined  as  a  deficiency  or 
shortcoming  in  existing  policy,  organization,  training,  education,  equipment,  or  doctrine 
that  requires  actions  to  correct.  (OPNAVINST  3500.37B)  If  a  lesson  learned  submission 
meets  this  criterion,  the  Fleet  Management  Site  will  assign  a  RAP  Working  Group  to  this 
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item.  The  Management  site  is  responsible  for  tracking  and  updating  the  NLLDB  on  the 
status  of  the  remedial  action.  If  an  item  cannot  be  resolved  at  the  Management  Site  Level, 
the  item  will  be  forwarded  to  a  CNO  Executive  Panel.  Once  the  item  is  accepted,  the 
CNO  panel  is  responsible  for  tracking  the  action  taken,  but  it  is  still  the  Management 
Site’s  responsibility  to  update  the  item  in  the  NLLDB.  If  the  CNO  Executive  Panel 
rejects  the  item,  the  Management  Site  will  note  the  reason  in  the  database  and  cancel  the 
project  as  appropriate.  (OPNAVINST  3500.37B) 

Tactical  or  procedural  deficiencies  will  be  forwarded  to  Tactical 
Development  and  Evaluation  (TAC  D&E)  Steering  Committees  for  consideration. 
Approved  considerations  will  be  developed  into  new  Tactics,  Techniques  or  Procedures 
for  inclusion  into  the  Navy  Warfare  Publications  (NWP)  system. 


C.  SUBMISSION,  PUBLICATION,  AND  DISTRIBUTION 

A  Fleet  User  will  submit  a  Lesson  Learned  (LL)  by  means  of  a  Navy  Instructional 
Input  Program  (NITP)  message  to  the  Immediate  Supervisor  In  Command  (ISIC).  The 
ISIC  will  review  the  LL  for  accuracy  and  forward  it  to  the  theater’s  NLLS  Management 
Site.  The  NLLS  Management  Sites  are  located  as  follows: 

CINCLANTFLT  NLLS  Management  Site,  Norfolk,  Virginia 
CINCPACTFLT  NLLS  Management  Site,  San  Diego,  California 
COMUSNAVCENT  NLLS  Management  Site,  Tampa,  Florida 
COMSIXTHFLT  NLLS  Management  Site,  Gaeta,  Italy 
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COMSEVENTHFLT  NLLS  Management  Site,  Yokosuka,  Japan 

At  each  Fleet  Management  Site  (FMS)  the  NLLS  Data  Analyst  has  the  lesson 
learned  validated  by  Navy  Staff  personnel  or  Subject  Matter  Experts.  Once  validated,  the 
FMS  will  make  certain  that  the  Lesson  Learned  Submission  /  Summary  Report  is  in  the 
proper  format.  He  will  then  assign  appropriate  keywords  and  ensure  the  comments 
section  uses  standardized  terminology. 

The  validated  lesson  learned  is  forwarded  to  the  NLLS  Central  Site  at  the  Naval 
Warfare  Development  Command  in  Newport,  RI,  for  final  processing  and  uploading  to 
the  Lessons  Learned  SIPRNET  Web  Site.  Along  with  the  weekly  Web  site  update,  the 
lessons  learned  database  is  distributed  quarterly  on  a  NLLS  two-volume  CD-ROM. 

Lessons  learned  will  remain  in  the  active  database  until  it  is  determined  by  the 
theater  CINC  or  SMEs  that  they  are  no  longer  valid,  have  been  corrected,  or  are  no  longer 
relevant  to  current  Navy  procedure.  At  that  point,  but  not  later  than  two  years  from  the 
entry  date,  the  lesson  learned  will  be  moved  to  the  inactive  database. 
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III.  ORGANIZATIONAL  LEARNING  MODEL 

A.  INTRODUCTION 

Organizations  continually  strive  for  improved  systems  of  information  collection, 
management,  and  analysis.  The  introduction  of  and  rapid  advancements  in  availability  of 
computers  during  the  last  30  years  have  resulted  in  more  evolved  systems  for  acquiring, 
storing,  and  disseminating  information.  The  opportunities  presented  by  these  advances 
have  produced  widely  varying  benefits  for  organizations  seeking  to  increase  stakeholder 
value.  This  identification  of  knowledge  as  a  means  for  competitive  advantage  has  meant 
an  exponential  demand  for  information  systems.  Worldwide  expenditures  on  information 
technology  have  generated  significant  but  somewhat  varying  benefits  to  those  companies 
who  invest  in  information  technology  for  knowledge  benefits.  (Drucker,  1994)  This 
explosion  of  data  often  threatens  to  overwhelm  individuals  with  information.  Advances 
have  focused  on  making  us  masters  of  gathering  data,  but  not  on  turning  that  data  into 
knowledge. 

Companies  today  gather  increasing  amounts  of  information,  much  of  it  of 
debatable  utility.  All  too  often  the  question  asked  is,  “What  data  do  you  want  to  gather?” 
instead  of,  What  do  you  plan  on  doing  with  it?”  Advances  in  computer  systems  have 
made  gathering  and  storing  data  simple.  The  problem  now  is  finding  and  understanding 
the  information  that  resides  in  these  huge  data  warehouses. 

Thomas  Davenport  of  Anderson  Consulting  gives  a  good  example  of  this  data-to- 
knowledge  deficit.  He  writes  that  the  CIO  of  a  grocery  chain  highly  regarded  for  its  use 
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of  IT  once  confided  to  him  that  his  company  analyzed  at  most  only  two  percent  of  the 
data  it  collected.  Another  example  Davenport  cites  is  that  of  a  midwestem  grocery  chain 
that  finally  decided  to  throw  away  its  scanner  data.  The  data  “had  been  saved  for  years  in 
the  hope  that  it  would  someday  be  analyzed,  but  it  never  was.”  (Davenport,  1999) 

The  challenge  facing  today’s  IT  people  is  not  to  provide  technology  or  record 
data,  but  to  help  the  organization  make  use  of  this  information. 

B.  DATA  VS.  KNOWLEDGE 

Quality  information  is  as  critical  a  resource  in  the  military  as  the  ships  we  serve 
on.  But  what  exactly  is  information,  and  what  separates  quality  information  from  data? 
Do  we  know  what  kind  of  information  we  need,  and,  if  so,  how  do  we  recognize,  capture, 
and  then  pass  it  on  to  others? 

As  simple  as  it  may  seem,  it  is  important  to  understand  that  data,  information,  and 
knowledge  are  not  interchangeable.  Understanding  the  difference  between  the  three  and 
what  you  can  and  can’t  get  from  each  can  mean  the  difference  between  success  and 
failure  in  individual  or  organizational  learning. 

1.  Data 

Data  can  be  defined  as  a  set  of  discrete,  objective  facts  about  events  (Davenport, 
1999).  In  the  organizational  context,  it  is  most  often  described  as  a  structured  record  of 
transactions.  The  example  Davenport  gives  involves  a  trip  to  the  local  gas  station.  The 
receipt  that  prints  out  after  the  customer  fills  the  tank  consists  of  data:  how  many  gallons 
the  customer  bought,  how  much  he  paid  for  each  gallon,  and  when  he  bought  the  gas.  The 
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data  by  itself  tells  the  reader  nothing  about  why  the  customer  chose  that  particular  gas 
station  over  another,  when  or  will  he  come  back  again,  or  how  well  or  poorly  that  gas 
station  is  managed.  (Davenport,  1999) 

Data  by  itself  can  be  evaluated  in  two  ways,  quantitatively  and  qualitatively. 
Quantitative  measures  would  include  how  much  the  data  costs  to  capture,  or  how  long  it 
takes  the  system  to  call  up  the  data.  Qualitative  measures  would  include  “timeliness, 
relevance,  and  clarity.”  (Davenport,  1999)  Do  we  have  access  to  it  when  we  need  it,  is  it 
what  we  need,  and  can  we  make  any  sense  out  of  it? 

Data  collecting  has  become  increasingly  efficient  in  many  organizations.  But  the 
problem  with  becoming  a  data  culture  is  twofold:  First,  the  more  data  you  collect  the 
harder  it  is  to  make  sense  out  of  it;  and  second,  there  is  no  inherent  meaning  in  data.  Data 
by  itself  tells  only  part  of  the  story. 

2.  Information 

One  definition  of  information  is  “data  that  has  been  put  into  a  meaningful  and 
useful  context  and  communicated  to  a  recipient  who  uses  it  to  make  decisions.”  (Burch 
and  Grudnitski,  1986)  Information  must  inform  and,  as  with  the  transfer  of  any  message, 
information  has  to  have  a  sender  and  a  receiver.  In  information,  it  is  the  receiver  who 
decides  what  is  really  information.  “A  memo  of  unconnected  ramblings  may  be 
information  to  the  sender  but  to  the  receiver  it  is  just  noise.”  (Davenport,  1999) 

Like  data,  information  can  be  evaluated  in  quantitative  and  qualitative  measures. 
Quantitative  measures  would  include  how  many  messages  we  sent  out  or  how  much  e- 


21 


mail  we  downloaded.  Qualitative  measures  would  include  whether  those  messages 
provided  any  useful  advice  on  how  we  should  do  our  jobs  today,  or  helped  us  arrive  at 
solutions  for  a  particular  problem. 

Davenport  contends  that  data  becomes  information  when  its  creator  adds  meaning 
or  adds  value  in  various  ways,  the  most  important  of  which  all  which  begin  with  the  letter 
C.  They  are: 

•  Contextualized :  Knowing  for  what  purpose  it  was  gathered. 

•  Categorized :  Establishing  the  units  of  analysis  or  key  components  of  the  data. 

•  Calculated-.  Analyzing  the  data  mathematically  or  statistically. 

•  Corrected :  Removing  errors  from  the  data. 

•  Condensed-.  Summarizing  the  data  in  a  more  concise  form. 

A  list  of  sports  scores  in  a  newspaper  is  an  example  of  data.  When  the  list  is 
framed  as  one  team  defeating  another,  information  results.  As  Peter  Drucker  said, 
“Information  is  data  endowed  with  relevance  and  purpose.”  (Drucker,  1992)  Sports  scores 
are  only  data,  but  when  a  reader  sees  the  win  or  loss  of  a  favorite  team,  he  has 
information. 

3.  Knowledge 

This,  however,  is  still  only  information.  To  gain  knowledge,  one  must  know  how 
to  use  this  information  -  or  in  this  case,  understand  how  the  game  is  played.  Someone 
who  did  not  understand  the  rules  of  golf,  for  example,  might  think  that  the  player  with  a 
score  of  83  did  better  than  the  player  with  a  score  of  72.  But  if  he  is  able  to  understand 
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and  internalize  the  information,  and  know  that  in  golf  the  lowest  score  wins,  then  he  can 
use  this  information  to  gain  knowledge  of  the  tournament  being  played. 

On  a  larger  scale  Davenport  defines  knowledge  in  this  way: 

Knowledge  is  a  fluid  mix  of  framed  experiences,  values,  contextual 
information,  and  expert  insight  that  provides  a  framework  for  evaluating 
and  incorporating  new  experiences  and  information.  It  originates  and  is 
applied  in  the  minds  of  knowers.  In  organizations,  it  often  becomes 
embedded  not  only  in  documents  or  repositories  but  also  in  organizational 
routines,  processes,  practices,  and  norms.  (Davenport,  1999) 

Just  what  characterizes  knowledge,  then,  is  not  so  easy  to  identify.  Following  the 

chain  we  set  up  earlier,  we  can  see  that  knowledge  is  derived  from  information,  as 

information  was  derived  from  data.  But  where  data  consists  of  a  computer  printout,  or 

information  a  newspaper  sports  page,  knowledge  resides  in  the  minds  of  individual. 

C.  ORGANIZATIONAL  LEARNING 

1.  Individual  Learning 

In  order  to  discuss  organizational  learning  theories,  we  first  must  discuss  the  role 
of  the  individual  in  that  learning.  M.  J.  Marquardt  points  out  that  “individual  learning  is 
needed  for  organizational  learning  since  individuals  form  the  units  of  groups  and 
organizations.”  Also  emphasizing  the  role  of  the  individual  is  Peter  Senge,  who  writes, 
“Organizations  learn  only  through  individuals  who  learn.  Individual  learning  does  not 
guarantee  organizational  learning,  but  without  it  no  organizational  learning  occurs.” 
(Senge,  1990) 
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Placing  somewhat  less  importance  on  the  role  of  the  individual  in  organizational 
learning  is  D.  H.  Kim,  who  argues,  “Organizational  learning  is  much  more  complex  and 
dynamic  than  a  mere  magnification  of  individual  learning.  ...  The  level  of  complexity 
increases  tremendously  when  we  go  from  a  single  individual  to  a  large  collection  of 
diverse  individuals.”  (Kim,  1993) 

Regardless  of  where  these  authors  stand  on  the  continuum  of  the  importance  of 
individual  learning  in  organizational  learning,  they  are  in  general  agreement  that 
individual  learning  is  the  starting  point  for  organizational  learning  to  take  place.  Without 
it,  organizations  as  entities  would  be  unable  to  learn  on  their  own. 

2.  Organizational  Learning 

Organizational  Learning  is  a  concept  that  has  gained  widespread  interest  in  both 
the  business  and  academic  communities.  From  the  business  side,  the  writings  focus  on 
paying  attention  to  the  concept  of  organizational  learning  in  order  to  provide  a  sustainable 
advantage  over  corporate  competitors.  In  the  academic  community,  the  writings  take  a 
procedural  approach,  looking  at  “detecting  and  correcting  errors”  or  “encoding  inferences 
from  history  into  routines  that  guide  behaviors.”  (Argyris  and  Schon,  1978;  Levitt  and 
March,  1 988)  Though  they  come  at  it  from  slightly  different  perspectives,  both  groups 
would  agree  that  organizational  learning  makes  an  organization  more  effective,  profitable, 
and  adaptable  to  change.  Because  of  this,  the  organization  should  ensure  processes  are  in 
place  that  allow  individual  and  organizational  alike  to  leam. 
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The  military  is  not  oblivious  to  this  realization  and,  in  fact,  takes  great  pains  to 
improve  on  detection  and  correction  of  errors.  Still,  organizational  learning  is  the  result  of 
a  wide  range  of  factors  and  influences  that  include  structure,  environment,  culture,  and 
technology.  The  purpose  of  reviewing  different  organizational  learning  models  is  to 
identify  those  factors  that  may  have  significance  to  a  military  organization.  As  General 
Gordon  R.  Sullivan  (U.S.  Army)  said: 

“Our  task  is  not  to  make  perfect  plans... our  task  is  to  create  organizations 
that  are  significantly  flexible  and  versatile  that  they  can  take  our  imperfect 
plans  and  make  them  work  in  execution  ”  (Sullivan  and  Harper,  1996) 

3.  Goals  of  NLLS  Organizational  Learning 

The  goal  of  the  Navy  Lessons  Learned  System  is  an  effort  by  the  organization  to 
promote  Navy-wide  learning  through  the  collection  and  dissemination  of  all  significant 
lessons  learned,  summary  reports,  and  port  visit  reports,  culled  from  the  day-to-day 
maritime  operations  that  are  routinely  undertaken.  The  Navy  Lessons  Learned  System  is 
designed  to  be  the  singular  Navy  Program  for  the  collection,  validation,  and  distribution 
of  unit  feedback  as  well  as  for  the  correction  of  problems  identified  from  fleet  operations. 
(OPNAVTNST  3500.37B)  The  goal  of  the  system  is  to  promote  organizational  learning  by 
ensuring  we  leam  lessons  once. 

D.  TYPES  OF  ORGANIZATIONAL  LEARNING 

In  this  thesis  I  focus  primarily  on  the  Models  of  Organizational  Learning 
presented  by  George  Huber  and  by  Chris  Argyris  and  Donald  Schon.  Huber  views 
organizational  learning  from  a  systems  perspective  primarily  concerned  with  the 
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acquisition,  distribution,  and  interpretation  of  information  from  the  environment.  Much  of 
his  writing  emphasizes  the  need  for  the  organization  to  provide  conduits  for  the 
distribution  of  information.  Argyris  and  Schon  place  greater  emphasis  on  the  individual. 
They  contend  that  the  way  to  change  organizations  is  to  change  the  way  individuals 
respond  to  detecting  and  correcting  errors. 

1.  Huber’s  Organizational  Learning  Model 

In  his  literature  review  on  organizational  learning,  Huber  describes  the  following 
four  processes,  or  constructs,  that  contribute  to  organizational  learning. 

a.  Knowledge  Acquisition 

Learning  occurs  when  an  organization  acquires  knowledge.  This 
knowledge  can  be  acquired  intentionally  by  such  means  as  organizational  experiments, 
which  result  in  focused  learning,  or  unintentionally  and  unsystematically,  which  results  in 
“haphazard  or  multi  faceted”  learning.  (Huber,  1991)  If  the  organization  facilitates 
experimental  learning,  the  focus  should  be  on  increasing  the  accuracy  of  feedback 
generated.  Also,  increased  emphasis  should  be  placed  on  determining  the  cause  and  effect 
relationships  between  actions  and  outcomes. 

Other  methods  of  knowledge  acquisition  include  “scanning  the 
environment”  for  knowledge  or  facts,  using  information  systems  to  store,  administer,  and 
retrieve  this  information,  and  the  grafting  of  new  members  who  “possess  knowledge  not 
previously  available  within  the  organization.”  (Huber,  1991) 
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b.  Information  Distribution 

According  to  Huber,  Information  Distribution  is  a  determinant  of  both  the 
“occurrence  and  breadth”  of  organizational  learning.  (Huber,  1991)  This  distribution  is 
the  process  by  which  an  organization  shares  information  among  its  members,  thereby 
promoting  knowledge.  The  more  widely  information  is  distributed  in  the  organization,  the 
more  varied  sources  for  it  to  continue  to  exist,  thus  making  retrieval  efforts  more  likely  to 
be  successful.  Brown  and  Duguid  concur  with  this,  stating  that  a  majority  of  learning 
takes  place  in  informal  “communities  of  practice”  by  members  sharing  stories  or 
anecdotes  of  work.  The  greater  the  sharing  or  distribution  of  information  within  the 
entity,  the  greater  the  organizational  learning  that  takes  place.  (Brown  and  Duguid,  1991) 

c.  Information  Interpretation 

Although  information  has  been  distributed  throughout  the  organization, 
the  intended  receivers  still  must  interpret  it.  Factors  affecting  the  interpretation  of 
information  include  the  receiver’s  cognitive  map,  or  frame  of  reference.  Researchers  have 
established  that  cognitive  maps  can  vary  greatly  across  units  within  the  organization. 
(Walker,  1985;  Kennedy,  1983  et  al)  Because  belief  structures  are  automatically  applied 
to  any  incoming  information,  understanding  this  interaction  between  cognitive  maps  and 
message  interpretation  is  important  to  understanding  how  organizations  learn. 

To  aid  in  this  information  interpretation,  the  greater  the  media  richness, 
the  greater  the  possibility  that  the  meaning  intended  by  the  sender  will  be  realized  by  the 
receiver  (Huber,  1991).  Still,  the  distribution  and  media  richness  must  be  tempered, 
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because  interpretation  across  organizational  units  is  diminished  if  the  amount  of 
information  exceeds  the  units’  capacity  to  process  it.  Information  overload  will  likely 
result,  with  filters  being  applied  and  little  or  no  information  being  processed. 

d.  Organizational  Memory 

Organizational  Memory  refers  to  the  way  an  organization  stores  its 
information.  It  can  be  hard  information,  which  is  stored  in  standard  operating  procedures, 
routines,  and  scripts,  or  it  can  be  soft  information,  which  is  corporate  knowledge, 
experiences,  or  practices.  For  an  organization  to  learn,  it  is  necessaiy  that  its  members  be 
able  to  access  the  information  stored  or  possessed  by  other  members.  The  greater  the 
specialization  or  departmentalization  of  an  organization,  the  greater  the  chance  members 
will  not  know  what  other  members  know.  In  order  for  an  organization  to  demonstrate 
learning,  learned  behavior  must  first  be  stored  in  memory  and  then  brought  forth  from 
memory.  Having  the  information  tucked  away  and  inaccessible  will  discourage  or  defeat 
learning  from  taking  place.  (Huber,  1991) 

2.  Argyris  and  Schon’s  Organizational  Learning  Model 

In  their  book,  Organizational  Learning:  A  Theory >  of  Action  Perspective,  Chris 
Argyns  and  Donald  Schon  make  a  connection  between  the  learning  of  the  individual  and 
the  learning  of  the  organization.  They  say  the  way  to  change  organizations  is  primarily 
through  changing  individual  actions.  This  book  introduced  the  concepts  of  single,  double, 
and  deutero  learning,  to  describe  the  way  organizations  learn  by  the  detection  and 
correction  of  errors. 
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a.  Single  Loop  Learning 

As  each  member  is  assimilated  into  an  organization  he  or  she  is 
indoctrinated  through  a  process  of  socialization  to  a  rule-governed  behavior  appropriate 
to  the  organization.  As  members  carry  out  the  practices  common  to  the  organization,  they 
manifest  a  kind  of  knowledge.  (Argyris  and  Schon,  1978)  This  reveals  itself  in  each 
member  as  his  or  her  own  image,  or  map,  of  the  organization.  Taken  as  a  whole,  this 
results  in  organizational  maps  that  are  the  shared  description  of  the  organization 
individuals  have  constructed.  This  shared  description,  or  map,  can  include  standard 
operating  procedures,  work  flow  diagrams,  and  any  “actual  patterns  of  activity”  that  are 
“guides  to  future  action.”  (Argyris  and  Schon,  1978) 

At  times  there  are  inputs  to  individual  members  that  run  counter  to 
organizational  norms.  When  there  is  a  mismatch  of  expectation  (detection  of  error), 
members  seek  to  align  activities  to  bring  expectations  back  into  norm  (correction  of 
errors).  This  is  an  example  of  single  loop  learning: 

Members  of  the  organization  respond  to  changes  in  the  internal  and 
external  environments  of  the  organization  by  detecting  errors  that  they 
then  correct  so  as  to  maintain  the  central  features  of  the  organization. 

(Argyris  and  Schon,  1978) 

Single  loop  organizational  learning  occurs  when  errors  are  detected  and 
corrected,  but  the  underlying  norms  and  polices  of  the  organization  remain  unchanged. 
Argyris  and  Schon’s  example  is  that  of  a  thermostat.  When  the  temperature  drops  below  a 
certain  set  value,  the  heater  turns  on.  The  thermostat  makes  no  attempt  to  discover  why 
the  temperature  has  fallen  below  the  prescribed  norm. 
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b.  Double  Loop  Learning 

Single  loop  learning  works  well  when  the  organization  is  concerned 
primarily  with  achieving  existing  goals,  but  there  are  many  cases  when  error  correction 
requires  that  organizational  norms  themselves  be  modified.  (Argyris  and  Schon,  1978) 
This  higher  learning  requires  an  organization  to  understand  its  environment  in  order  to 
increase  the  range  of  options  available.  For  members  to  simply  keep  on  doing  what  they 
already  know,  but  better,  is  not  enough  to  correct  the  error.  In  double  loop  learning,  in 
addition  to  the  detection  and  correction  of  errors,  the  organization  is  actively  involved  in 
the  “significant  restructuring  and  configuration  of  corporate  norms.”  (Argyris  and  Schon, 
1978) 


Double  loop  learning  is  those  sorts  of  organizational  inquiries  that 
resolve  incompatible  organizational  norms  by  setting  new  priorities  and 
weighting  of  norms,  or  by  restructuring  the  norms  themselves  together 
with  associated  strategies  and  assumptions.  (Argyris  and  Schon,  1978) 

Double  loop  learning  would  occur  if  the  thermostat  could  determine  that 
the  temperature  dropped  because  the  window  is  open,  and  then  close  it. 


c.  Deutero-Learning 

It  has  become  apparent  to  many  organizations  that  even  changing  the 
underlying  norms  when  a  mismatch  occurs  is  not  enough.  In  order  for  a  company  to  be 
continually  innovative,  it  needs  to  learn  how  to  carry  out  single  and  double  loop  learning. 
This  type  of  “learning  how  to  leam”  is  called  deutero-leaming  or  second-order  learning. 
(Argyris  and  Schon,  1978) 
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When  a  company  engages  in  deutero-leaming,  it  discovers  what  occurred 
to  facilitate  or  inhibit  learning  and  invents  new  strategies  for  learning.  (Argyris  and 
Schon,  1978) 

In  single  loop  learning,  individuals  are  concerned  with  the  effectiveness  of 
existing  norms  and  procedures  without  questioning  underlying  procedures.  In  double  loop 
learning,  they  follow  error  detection  by  questioning  the  organizational  norms  themselves 
for  error  correction.  If  the  organization  becomes  concerned  with  discovering  how  it 
leams,  then  it  is  engaged  in  deutero-leaming. 

E.  INFLUENCES  ON  ORGANIZATIONAL  LEARNING 

The  previous  sections  highlighted  two  views  on  organizational  learning.  Though 
slightly  different  in  their  focus,  they  addressed  the  topic  from  a  perspective  that  the 
learning  of  an  organization  was  based  either  in  part  or  in  whole  on  the  knowledge  or 
actions  of  the  individuals  as  an  aggregate.  While  the  individual  is  certainly  in  the 
forefront,  Marlene  Fiol  and  Maijorie  Lyles  inform  us  there  are  other  factors,  such  as  the 
structure,  environment,  and  culture,  that  influence  the  probability  that  learning  takes 
place. 

1.  Structure 

According  to  Fiol  and  Lyles,  a  centralized  mechanistic  structure  tends  to  reinforce 
past  behaviors.  A  mechanistic  organization  is  thus  only  capable  of  Argyris  and  Schon’ s 
single  loop  learning.  However,  an  organic,  more  decentralized  structure  promotes  the 
shifts  of  beliefs  and  actions  that  allow  for  double  loop  learning.  (Fiol  and  Lyles,  1985)  To 
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encourage  learning,  management  must  therefore  move  away  from  mechanistic  structures 
that  encourage  conformance  to  existing  norms  toward  the  organic,  flexible  structures  that 
encourage  reflective  action-taking.  (Fiol  and  Lyles,  1985) 

2.  Environment 

While  Fiol  and  Lyles  acknowledge  there  is  some  disagreement  over  the  different 
viewpoints  on  Organizational  Learning,  they  point  out  that  there  is  much  more  consensus 
on  the  importance  of  the  environment  to  the  organization.  The  “ultimate  aim  of  the 
organization”  is  long-term  survival,  and  to  accomplish  this  “companies  align  with  their 
environment”  to  remain  competitive  and  innovative.  This  alignment  implies  that  the  firm 
must  have  the  potential  to  learn,  unlearn,  or  relearn  based  on  its  past  behaviors.  (Fiol  and 
Lyles,  1985) 

This  learning  is  conditional,  however.  If  the  internal  or  external  environment  is 
too  complex  for  the  organization  to  handle,  no  learning  will  take  place.  In  order  to  leam, 
there  must  be  both  “change  and  stability.”  Too  much  stability  causes  the  organization  to 
be  static  or  dysfunctional.  Too  much  change  and  the  organization  is  unable  to  map  the 
environment.  Learning  requires  a  balance  between  uncertainty  and  constancy.  (Fiol  and 
Lyles,  1985) 

3.  Culture 

Culture  can  be  defined  as  an  organization’s  values,  beliefs,  and  norms,  manifested 
as  symbols,  rituals,  and  language.  Thus,  culture  will  influence  organizational  action¬ 
taking.  (Fiol  and  Lyles,  1985)  It  is  these  norms,  or  established  patterns  of  behavior,  that 
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can  encourage  or  inhibit  learning  and  partially  determine  strategy  and  the  direction  of 
organizational  change.  Changing  the  capacity  of  an  organization  to  learn  thus  often 
involves  the  changing  or  restructuring  of  the  norms  and  belief  systems. 
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IV.  USE  OF  TECHNOLOGY  IN  A  LEARNING  ORGANIZATION 

A.  INTRODUCTION 

Each  day  we  are  presented  with  the  latest  and  greatest  advances  in  information 
technology,  and  all  are  being  publicized  as  critical  to  an  organization.  Brand  new 
technologies  such  as  data  mining,  intranets,  video  conferencing,  and  distance  learning  are 
being  pushed  as  solutions  for  the  challenges  we  face  as  we  leave  the  Information  Age  and 
enter  the  Knowledge  Age. 

Today,  the  Internet  presents  us  with  almost  limitless  capabilities.  E-mail, 
discussion  groups,  and  database  searching  are  just  a  few  uses  of  the  technology.  The 
ability  to  reach  almost  anyone,  at  any  time,  presents  us  with  an  extraordinary  opportunity 
to  expand  the  organization’s  learning  potential. 

Still,  not  all  technical  experts  and  academic  scholars  are  in  agreement  on  the 
benefits  realized  from  the  increasing  amounts  of  investment  in  Information  Technology. 
Karl  Sveiby  contends  that  the  confusion  between  knowledge  and  information  has  caused 
managers  to  sink  billions  of  dollars  in  information  technology  ventures  that  have  yielded 
marginal  results.  (Sveiby,  1997)  Erik  Bmjolfsson,  a  professor  of  Information  Systems  at 
MIT  Sloan  School,  notes  in  Information  Week:  “The  same  dollar  spent  on  the  same 
system  may  give  competitive  advantage  to  one  company  but  only  expensive  paperweights 
to  another.”  (Sept  9,  1996) 
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In  this  chapter  we  will  look  at  the  emerging  issue:  How  can  organizations  use 
Information  Technology  to  support  a  Learning  Organization  and  improve  organizational 
performance? 

B.  BUILDING  A  LEARNING  ORGANIZATION 

The  1990s  have  been  characterized  by  a  growing  interest  in  the  learning  process. 
This  has  been  caused  by  a  widespread  belief  that  learning  and  innovation  are  essential  for 
an  organization  to  survive.  As  a  consequence  of  a  rapidly  changing  environment, 
organizations  are  “going  out  of  business  everyday  because  they  have  failed  to  adapt  to 
change  or  they  have  adapted  too  slowly.”  (Ackoff,  1981)  R.  L.  Ackoff,  among  others, 
believes  that  businesses  that  fail  to  become  learning  organizations  will  not  survive. 
Echoing  these  thoughts  is  Peter  Senge,  who  remarks,  “The  rate  at  which  organizations 
leam  may  become  the  only  sustainable  source  of  competitive  advantage.”  (Senge,  1990) 

Though  the  military  does  not  have  to  worry  about  going  out  of  business,  the 
lessons  of  Ackoff  and  Senge  are  more  relevant  than  one  would  first  think.  There  is  a 
common  axiom  that  the  militaries  of  the  world  adopt  doctrine  to  re-fight  and  win  the  last 
war.  For  example,  the  French  built  the  Maginot  Line  to  stop  a  German  offensive  based  on 
correcting  deficiencies  of  trench  warfare  identified  in  the  latter  days  of  WWI.  Iraq  built 
vast  static  defensive  positions  to  counter  mechanized  coalition  forces  in  Operation  Desert 
Storm  as  a  result  of  earlier  success  against  waves  of  dismounted  Iranian  Infantry.  Also, 
the  United  States  Army’s  attempts  to  overthrow  Adid  in  Somalia  were  based  on  tactics 
developed  in  Vietnam  to  fight  a  guerrilla  force,  not  on  the  Clan  relationship  that  actually 
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existed.  There  is  no  greater  rapidly  changing  environment  than  war,  and  the  price  of 
failure  to  adapt  can  mean  the  loss  of  life.  Taking  Ackoff  s  words  to  heart,  we  will  look  at 
what  comprises  a  Learning  Organization  and  how  we  can  introduce  organizationional 
learning  into  our  organization. 

Before  we  move  onto  the  process  of  building  a  learning  organization,  we  must 
first  define  what  a  learning  organization  is  and  how  it  differs  from  organizational 
learning.  David  Garvin,  noting  that  although  “surprisingly,  a  clear  definition  of  a  learning 
organization  has  proved  elusive  over  the  years,”  gives  us  perhaps  the  most 
comprehensible  definition: 

A  learning  organization  is  an  organization  skilled  at  creating,  acquiring, 

and  transferring  knowledge,  and  modifying  its  behavior  to  reflect  new 

knowledge  and  insights.  (Garvin,  1993) 

It  is  worth  noting  that  some  of  the  published  literature  uses  the  terms  Learning 
Organization  and  Organizational  Learning  interchangeably.  For  our  purposes,  we  will 
follow  Michael  Marquardt’s  argument  that  there  is  a  distinct  difference  between  the  two 
and  that  it  can  best  be  explained  as  a  “process”  vs.  “product”  argument.  The  learning 
organization  is  the  desired  end  product.  Organizational  Learning  is  the  cognitive  process 
required  to  raise  the  capacity  of  the  total  organization  towards  that  result.  (Marquardt, 
1996) 

1.  Five  Building  Blocks  Of  A  Learning  Organization 

Garvin  believes  that  a  learning  organization  has  members  experienced  at  five 
main  activities.  They  are:  systematic  problem  solving,  experimentation  with  new 
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approaches,  learning  from  their  own  experience  and  past  history,  learning  from 
experiences  and  best  practices  of  others,  and  transferring  knowledge  quickly  and 
efficiently  throughout  the  organization.  (Garvin,  1993)  We  will  look  at  these  individually 
to  see  if  and  how  they  relate  to  the  Navy  and  the  Navy  Lessons  Learned  System. 

a.  Systemic  Problem  Solving 

Systemic  problem  solving  entails  an  organization’s  use  of  scientific 
methods  for  diagnosing  problems,  rather  than  guesswork.  By  using  simple  statistical 
tools,  such  as  histograms  and  Pareto  charts,  organization  members  will  be  using  data 
instead  of  their  own  assumptions  to  make  decisions.  The  idea  behind  this  is  to  get 
organizations  to  continually  ask  themselves,  “How  do  we  know  that  is  true?”  A  common 
phrase  in  the  military  is,  “That’s  close  enough  for  government  work.”  But  if  learning  is  to 
take  place,  we  cannot  be  satisfied  until  we  are  certain  that  we  have  looked  for  the  best 
answer.  (Garvin,  1993) 

b.  Experimentation 

Experimentation  is  described  as  “the  systematic  searching  for  and  testing 
of  new  knowledge.”  (Garvin,  1993)  Testing  new  knowledge  differs  from  problem  solving 
in  the  fact  that  it  is  not  seeking  to  find  solutions  for  current  problems.  Instead,  it  is 
motivated  by  the  desire  to  expand  the  horizon.  Experimentation  for  Garvin  takes  two 
main  forms: 

•  Ongoing  Programs  that  are  a  continuing  series  of  small  experiments 
designed  to  produce  incremental  gains  in  knowledge. 
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•  Demonstration  Projects  that  are  more  complex  and  involve  holistic 
system  changes  introduced  at  a  single  site  that  have  the  goal  of 
developing  new  organizational  capabilities. 

Experimentation  in  the  Navy  is  an  active  and  ongoing  concept.  A  major 
source  of  this  takes  place  at  the  Maritime  Battle  Center  at  the  Naval  Warfare 
Development  Command  in  the  form  of  Fleet  Battle  Experiments  (FBE).  A  FBE  is 
designed  as  an  experiment,,  not  an  exercise.  Its  goal  is  to  provide  new  doctrine  or  new 
insights  into  emerging  technology  in  an  operational  environment.  This  will  in  turn 
generate  ideas  for  further  warfare  concepts,  or  subsequent  FBEs.  (NWDC  Public  Affairs, 
2000) 


c.  Learning  From  Past  Experience 

Companies  must  review  their  past  successes  and  failures  and 
systematically  assess  them.  Lessons  must  be  recorded  in  an  easily  available  form  so 
employees  have  access  to  them.  (Garvin,  1993) 

Garvin  cites  the  example  of  Boeing’s  work  to  overcome  the  design  and 
production  problems  it  experienced  with  the  737  and  747  by  going  back  and  comparing 
that  process  with  the  development  process  of  the  707  and  727,  two  of  the  companies  most 
profitable  planes.  A  three-year  study  resulted  in  a  one-inch  thick  book  of  lessons  learned 
and  recommendations.  Several  members  of  this  team  were  then  transferred  to  the  767 
start  up,  where  they  produced  the  most  “successful,  error  free  launch  in  Boeing’s  history.” 
(Garvin,  1993) 
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British  Petroleum  takes  the  learning  process  a  step  further.  The  company 
has  established  a  post-project  appraisal  unit  that  reviews  major  projects,  writes  up  case 
studies,  and  derives  lessons  that  are  then  incorporated  into  the  organization’s  planning 
guidance.  The  bulk  of  this  unit’s  time  is  spent  in  the  field  interviewing  managers  for  their 
analysis.  (Garvin,  1993) 

This  part  of  Garvin’s  Learning  Organization  most  closely  resembles  the 
Navy  Lessons  Learned  System,  with  a  couple  of  exceptions.  Unlike  the  Boeing  example, 
the  Navy  Lessons  Learned  System  does  not  currently  do  a  trend  analysis  on  the  data. 
Second,  where  BP  has  an  active  collection  process,  the  Navy  relies  on  a  passive  input 
system  for  collection. 

d.  Learning  From  Others 

Learning  from  others  entails  looking  outside  one’s  own  immediate 
environment.  The  idea  is  that  the  observations  of  other  companies  can  at  times  serve  as  a 
catalyst  for  new  thinking  within  your  own  organization.  The  commonly  accepted  term  is 
“benchmarking,”  an  “ongoing  investigation  and  learning  experience  that  ensures  that  the 
best  industry  practices  are  uncovered,  analyzed,  adopted,  and  implemented.”  (Garvin, 
1993)  Still  another  way  of  securing  an  outside  opinion  is  getting  feedback  from  your 
customers  or  observing  them  actively  using  your  product. 

While  the  relevance  of  these  ideas  to  the  military  may  be  seem  debatable, 
the  idea  behind  them  is  not.  In  order  to  leant,  an  organization  has  to  be  receptive  to  new 
ideas.  By  being  less  defensive  when  criticism  is  offered,  we  allow  new  ideas  to  surface. 
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e.  Transferring  Knowledge 

In  order  to  leam,  the  sharing  of  ideas  and  knowledge  must  be  done  quickly 
and  efficiently  throughout  the  organization.  (Garvin,  1993)  As  a  means  of  accomplishing 
this,  Garvin  lists  a  number  of  available  methods,  each  with  its  own  strength  and 
weakness. 

Perhaps  the  easiest  and  most  prevalent  means  of  sharing  is  through  the  use 
of  reports  or  tours.  Garvin  believes,  however,  that  despite  their  popularity,  reports  and 
tours  are  a  cumbersome  way  of  transferring  knowledge.  Trying  to  absorb  facts  from  a 
written  report  or  from  seeing  a  complex  concept  demonstrated  is  fairly  difficult.  It  is 
much  easier  and  more  effective  to  actively  experience  it.  From  this  observation,  Garvin 
notes  that  personnel  rotation  programs  are  one  of  the  most  powerful  methods  of 
transferring  knowledge.  (Garvin,  1993)  When  expertise  is  held  locally,  those  people  who 
are  fortunate  enough  to  have  daily  contact  benefit  tremendously.  By  transferring  the 
expert  to  different  parts  of  the  organization,  companies  are  able  to  spread  the  knowledge 
around. 

Another  extremely  effective  method  of  transferring  knowledge  is  through 
the  use  of  education  and  training  programs  that  focus  on  problem  solving.  To  be 
effective,  the  organization  must  follow  through  and  ensure  that  this  training  is  tied 
directly  to  implementation.  (Garvin,  1993)  In  addition,  the  use  of  problem  solving 
techniques  can  be  increased  if  employees  feel  that  their  ideas  will  be  listened  to.  One  way 
of  doing  this  is  through  the  establishment  of  an  awards  system.  When  employees  know 
that  their  ideas  are  being  evaluated  and  that  they  have  buy-in  from  the  top,  the 
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organization  will  have  greater  success  in  implementing  new  ideas  and  changing  the 
entrenched  status  quo. 

C.  MEASURING  LEARNING 

Attempts  to  measure  learning  can  trace  their  roots  back  to  the  early  part  of  the  20th 
century  when  businesses  noticed  that  production  costs  typically  fell  with  a  corresponding 
increase  in  cumulative  volume.  This  soon  became  known  as  the  learning  curve.  Learning 
curves  in  the  80  to  85  percent  range  of  their  projected  level  were  not  uncommon.  (Garvin, 
1993) 

For  organizations  trying  to  measure  learning,  however,  these  curves  are 
incomplete.  They  focus  on  a  single  output,  such  as  price,  and  do  not  address  other  values, 
such  as  quality  or  new  product  introductions.  (Garvin,  1993)  In  order  to  measure 
learning,  a  new  framework  had  to  be  developed. 

To  track  this  progress,  Garvin  believes  that  organizational  learning  usually 
proceeds  through  three  overlapping  and  measurable  stages: 

•  Cognitive  Stage:  Members  of  the  organization  are  exposed  to  new  ideas  and 
begin  to  think  differently. 

•  Behavioral  Stage:  Employees  begin  to  internalize  new  insights  and  alter  their 
behavior. 

•  Performance  Improvement  Stage:  Changes  in  behavior  lead  to  measurable 
improvements  in  results. 
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At  the  cognitive  level,  surveys  and  questionnaires  are  used  to  test  the  depth  of 
understanding  and  to  see  if  some  of  the  meanings  or  terms  are  still  unclear.  (Garvin, 
1993) 

To  evaluate  changes  in  behavior,  in  addition  to  the  surveys,  direct  observation 
must  be  made  of  the  employees  in  action.  This  can  been  done  by  implementing  quality 
control  recordings  of  phone  calls,  placing  orders  to  customer  service  desks,  or  by  use  of 
“mystery  shoppers”  to  sample  the  product  at  particular  stores. 

Finally,  a  comprehensive  learning  audit  is  undertaken  to  measure  and  assess  the 
cognitive,  behavioral,  and  performance  changes  to  ensure  the  efforts  have  produced 
results.  (Garvin,  1993) 

D.  STEPS  TOWARDS  BECOMING  A  LEARNING  ORGANIZATION 

Transforming  an  organization  into  a  learning  organization  is  not  an  easy 
undertaking.  It  requires  a  commitment  from  everyone  involved,  and  the  results  may  be 
slow  in  presenting  themselves.  Still,  there  are  a  number  of  concepts  the  organization  can 
embrace  immediately  to  expedite  the  process. 

The  first  step  is  to  foster  an  environment  that  is  conducive  to  learning.  The 
organization  should  allow  time  for  reflection  and  analysis.  When  members  are  not 
“hamed  or  rushed,”  they  will  have  time  to  identify  cause  and  effect  relationships,  or 
assess  current  work  systems,  and  learning  will  occur.  (Garvin,  1993) 

The  creation  of  learning  forums,  which  are  programs  or  events  designed  with 
explicit  goals  in  mind,  is  another  step.  (Garvin,  1993)  Asking  provocative  questions. 
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talking  to  customers,  suppliers,  or  outside  experts  can  open  up  the  organization  to  new 
ideas.  Turning  to  non-traditional  sources  for  information  or  seeking  opinions  from  people 
one  would  not  normally  consult  can  help  generate  new  ideas.  Division  officers  can  seek 
input  from  members  who  work  for  them,  asking  them  to  come  up  with  a  different  way  of 
doing  business.  Anyone  who  hears  the  phrase  “but  that’s  the  way  it’s  always  been  done” 
should  immediately  question  its  validity.  By  employing  a  critical  systematic  thinking 
process,  members  of  the  organization  can  identify  previously  held  assumptions  and 
provoke  feedback  loops,  which  will  provide  and  encourage  learning. 

The  last  and  most  important  step  is  to  open  up  information  flows.  “Boundaries 
inhibit  the  flow  of  information,”  and  knowledge  gained  during  routine  operations  will 
become  compartmentalized,  thereby  reinforcing  preconceived  conceptions  or  habits. 
(Garvin,  1993)  Opening  flows  can  be  done  in  a  number  of  ways.  Regularly  scheduled 
conferences,  meetings,  or  teams  that  are  designed  to  cross  levels  in  the  organization  are 
just  a  few.  The  greatest  tools  to  cross  platforms  and  connect  people  are  the  rapidly 
increasing  capabilities  of  Information  Technology,  which  can  be  used  to  generate, 
capture,  and  share  knowledge  throughout  the  organization. 

E.  SUMMARY 

This  chapter  provides  a  frame  of  reference  in  which  to  understand  the  processes 
that  make  up  organizational  learning  and  some  general  concepts  and  definitions  that  will 
be  used  in  later  analysis.  The  importance  of  the  individual  as  a  basis  for  organizational 
learning  was  explored  and  discussed,  as  were  several  methods  of  acquiring  knowledge. 
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Equal  importance  was  given  to  external  factors  that  are  critical  in  understanding  how  the 
organization  learns.  Structure,  environment,  and  culture  all  affect  the  individual’s  frame 
of  reference  and  the  range  of  potential  behaviors  that  may  result. 
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V.  INFORMATION  TECHNOLOGY  IN  OTHER  ORGANIZATIONS:  THE 
CENTER  FOR  ARMY  LESSONS  LEARNED  AND  HAITI,  A  CASE 

STUDY 


A.  INTRODUCTION 

It  has  been  shown  that  organizational  learning  occurs  when  the  knowledge  and 
experience  that  reside  in  the  organization’s  members  are  shared  throughout  the 
establishment.  In  order  for  learning  to  occur  in  this  way,  however,  the  organization  relies 
on  its  people  and  groups  to  be  agents  for  this  transfer  of  knowledge.  As  the  organization 
matures,  this  knowledge  is  built  into  the  culture,  structure,  and  memory  of  the 
organization  so  that  even  though  people  leave  the  organization,  the  knowledge  remains. 

Organizations  do  not  have  to  rely  solely  on  their  members’  interactions  with  one 
another  for  sharing  to  occur.  They  can  enhance  this  transfer  of  knowledge  through  the  use 
of  Information  Technologies. 

IT’s  increasingly  widespread  usage  can  contribute  to  the  growth  of  organizational 
knowledge  in  two  ways.  First,  it  furnishes  an  extremely  efficient  way  to  collect  the 
information  that  resides  in  its  members,  even  those  who  reside  in  its  farthest  node. 
Second,  it  provides  the  members  with  an  easily  accessible  means  of  retrieval  of  this 
information  through  the  use  of  Internet  and  intranets,  distributed  databases,  or  knowledge 
portals.  Storing  the  organization’s  knowledge  base  using  IT  systems  gives  the  added 
benefit  of  rapid  modification  of  information  as  needed.  For  comparison  to  the  Navy 
Lessons  Learned  System,  we  will  look  at  another  organization,  the  Center  for  Army 
Lessons  Learned  (CALL).  Particular  emphasis  will  be  placed  on  CALL’S  efforts  to 
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implement  Information  Technology  in  order  to  become  a  more  effective  learning 
organization. 


B.  HISTORY 

Like  the  Navy,  in  the  mid  1980s  the  Army  leadership  realized  that  despite  the 
investment  in  the  National  Training  Center  (NTC),  there  was  no  method  to  capture  the 
lessons  learned  from  training  centers  in  the  Mojave  Desert.  (CALL,  97-13)  In  order  to 
accomplish  this,  in  1985  the  Army  created  the  Center  for  Army  Lessons  Learned  (CALL). 
CALL’S  initial  focus  was  on  the  continental  United  States  (CONUS)  units  as  they  rotated 
through  NTC’s  desert  training.  Today  CALL  operates  collections  centers  at  the  Joint 
Readiness  Training  Center  (JRTC)  for  light  forces  at  Fort  Polk,  LA,  the  Combat 
Maneuver  Training  Center  (CMTC)  at  Hohenfels,  GE,  and  the  Battle  Command  Training 
Program  (BCTP)  at  Fort  Leavenworth,  KS. 

C.  PROCESS 

The  CALL  staffs  are  continually  looking  out  to  identify  events  that  are  indicative 
of  new  or  future  missions  in  the  Army.  These  events  will  then  be  candidates  for  a 
Combined  Arms  Assessment  Team  (CAAT),  who  will  be  used  to  gather  data  for  lessons 
learned  and  organizational  knowledge  gained. 

Of  particular  recent  interest  to  CALL  were  the  contingency  peacekeeping 
operations  that  have  sprung  up.  Haiti  was  a  prime  example.  By  sending  a  team  of  experts 
to  observe  firsthand  the  new  situations,  CALL  hoped  to  document  and  develop  problem- 
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solving  strategies  on  the  spot.  (Henderson,  1998)  Of  particular  note,  historically  CALL’S 
approach  was  to  widely  deploy  observation  teams  with  little  effort  to  define  learning 
objectives.  “This  strategy  resulted  in  the  collection  of  massive  amounts  of  raw  data  that 
overloaded  the  Army’s  capacity  to  turn  it  into  useful  information.”  (Henderson,  1998) 
Today,  CALL  selects  events  for  observation  that  it  feels  has  a  “high  potential”  for 
providing  data  with  significant  value. 

Once  an  event  has  been  selected,  and  prior  to  deployment,  each  CAAT  team 
member  will  be  trained  as  a  “directed  telescope”  and  provided  a  detailed  set  of  questions 
about  events  they  are  to  observe  in  the  field.  (Henderson,  1998)  This  helps  to  keep  team 
members  focused  and  provides  a  structure  for  the  events  they  are  observing.  Collection 
personnel  deploy  with  sufficient  equipment  and  supplies  that  allow  them  to  electronically 
pass  observations  and  data  back  to  CALL  using  digital  equipment.  This  also  allows  team 
members  to  log  on  to  the  CALL  database  while  in  the  field  to  access  previous  lessons 
learned  information.  (CALL,  97-13). 
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D.  COLLECTION  TEAM  ORGANIZATION 


From  Ref.  [CALL,  97-13] 

CALL  collection  teams  are  composed  of  eight  to  twelve  expert  personnel  who  are 
tasked  from  various  units  in  the  Army.  The  majority  of  personnel  are  pulled  from  the 
schoolhouses  (TRADOCs)  and  are  selected  for  a  particular  skill  that  is  relevant  to  the 
knowledge  being  sought.  (Henderson,  1998)  The  person  in  charge  of  the  team  will 
usually  be  a  high  ranking  officer  (0-5  or  0-6)  who  is  also  tasked  from  the  FORSCOM 
Army.  To  assist  the  collection  team,  CALL  will  provide  three  people,  an  Operations 
Officer  (OPSO)  to  share  responsibilities  with  the  team  leader,  an  Operations  Non 
Commissioned  Officer  (OPSNCO),  and  a  civilian  analyst  who  is  assigned  to  the  team  but 
remains  at  Leavenworth. 
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E.  COLLECTION  PROCESS 


From  Ref.  [CALL,  97-13] 


Once  the  team  is  on  station,  the  members  join  an  army  unit  on  patrol  to  observe 
and  document  events  in  real  time.  It  is  important  to  note  that  CALL  observers  are  not 
evaluators.  The  CALL  personnel  are  attached  to  the  unit  and  become  an  integral  part  of 
the  operation.  (CALL,  97-13)  They  are  there  to  support  the  unit  as  well  as  observe. 

For  collection  purposes,  the  observed  information  must  be  operational  or  tactical 
in  nature.  Once  a  problem  is  identified,  the  CALL  team  follows  it  back  to  its  source, 
looking  to  identify  systematic  problems,  not  temporary  anomalies.  (Henderson,  1998) 
After  a  patrol  is  finished,  the  observations  will  be  verified  with  the  mission  commander 
and  discussed  further  with  the  other  observation  teams  members.  The  observations  are 
then  sent  to  an  analyst  at  Fort  Leavenworth  for  early  feedback.  The  analyst  will  discuss  it 
with  other  experts  and  send  back  either  more  questions  or  possible  solutions. 
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F. 


INFORMATION  TECHNOLOGY  TO  SUPPORT  LESSONS  LEARNED 


1.  Contingency  Operations 

CALL  and  CAAT  teams  effectively  utilize  IT  for  integrating  new  knowledge. 
Within  days  of  arriving  in  Haiti,  the  CAAT  teams  were  sending  raw  data  back  to  the 
analyst  via  satellite  at  the  rate  of  five  to  ten  observations  a  day.  (Henderson,  1 998)  These 
unprocessed  observations  were  posted  on  various  electronic  bulletin  boards  and  also 
electronically  distributed  to  a  network  of  appropriate  specialists  for  feedback.  This 
allowed  better  questions  to  be  devised,  and  after  discussion,  reflection,  and  feedback,  the 
new  information  was  edited  for  inclusion  into  the  electronic  organizational  memory,  or 
what  we  know  as  the  lessons  learned  database.  (Henderson,  1 998) 

To  supplement  the  observations,  videotapes  and  descriptions  of  what  happened 
were  sent  via  regular  mail.  With  these  were  diagrams  of  the  events  and  what  the 
observers  noted  were  possible  causes  and  consequences. 

The  team  returned  to  Fort  Leavenworth  after  two  months  of  patrols  in  Haiti.  A 
three-day  meeting  then  took  place,  during  which  they  went  through  the  observations, 
deciding  what  lessons  to  publish  in  an  “Initial  Impressions”  report.  This  report  was  then 
sent  out  to  thousands  of  ground  and  replacement  troops.  (Henderson,  1998) 

The  team  also  produced  over  100  vignettes  of  real  world  scenarios  for 
replacement  troops.  These  vignettes,  along  with  videos  of  actual  missions  and  the  lessons 
learned  information  gained,  were  used  to  provide  realistic  training  for  replacement  troops 
that  was  complete  “right  down  to  the  barking  dogs,  rotting  garbage,  and  belligerent 
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crowds.”  The  presentation  so  closely  approximated  conditions  the  soldiers  would 
encounter  that  CALL  was  credited  with  providing  for  a  seamless  troop  transition  as  the 
10th  Mountain  Division  was  replaced  by  the  25th  Infantry  Division  troops.  (Henderson, 
1998) 


2.  Combat  Training  Center  Operations 

Contingency  Operations  are  not  the  only  scenarios  that  CALL  monitors  for 
lessons  learned.  Ongoing  training  is  conducted  practically  every  day.  This  enables  Corps 
to  Squad  level  Army  units  to  fight  a  well-equipped  and  well-trained  enemy  in  a  variety  of 
terrains  that  range  from  barren  desert  to  densely  vegetated  woodlands.  (CALL,  97-13) 
This  training  takes  place  at  the  Combat  Training  Centers  (CTCs).  It  is  at  these  CTCs  that 
the  most  continuous  sources  of  observations  and  lessons  learned  are  generated,  and  under 
the  most  realistic  of  situations.  CALL  uses  these  exercises  and  lessons  learned  to  provide 
three  essential  products  to  the  rest  of  the  Army:  the  CTC  Quarterly  Bulletin,  the  CTC 
Trends  and  the  Topic  Newsletter. 

a.  CTC  Quarterly  Bulletin 

The  CTC  Quarterly  Bulletin  publishes  articles  on  techniques  and 
procedures  that  work.  It  is  authored  by  current  or  former  CTC  Observers,  and  its  audience 
is  primarily  units  that  are  scheduled  for  a  CTC  rotation.  (CALL,  97-13) 
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b.  CTC  Trends 

(1)  CTC  Trends  Bulletin.  CALL  receives  trends  and  associated 
TTPs  from  all  of  the  training  centers  on  a  regular  basis.  These  are  generated  from  unit 
rotations  at  the  CTCs.  The  CTC  branch  of  CALL  organizes  the  trends  and  publishes  a 
trends  bulletin  every  six  months.  (CALL,  97-13) 

(2)  CTC  Priority  Trends  Compendium.  All  of  the  recurring  trends 
and  associated  TTPs  for  each  CTC  are  compiled  into  a  compendium  of  priority  trends, 
which  are  published  annually.  The  compendium  also  contains  a  matrix  chart  that  shows 
the  number  of  times  per  quarter  that  a  particular  trend  was  documented  over  the  previous 
two  or  more  years.  (CALL,  97-13) 

(3)  CTC  Trends  Analysis.  The  CTC  trend  analysis  is  a  two-part 
product.  In  the  first  part,  for  each  CTC,  CALL  publishes  a  separate  analytical  review  of 
each  of  the  priority  trends  that  were  listed  in  the  CTC  Priority  Trends  Compendium.  The 
second  part  is  a  cross  CTC  analysis  of  all  CTCs.  (CALL,  97-13) 

c.  Topic  Newsletter 

The  Topic  Newsletter  picks  a  specific  subject  or  issue.  It  then  highlights 
the  areas  that  are  potential  problems  and  finally  provides  useful  tactics,  techniques,  and 
procedures  to  be  used  throughout  the  force. 

G.  INFORMATION  SYSTEMS  DIVISION 

The  Information  Systems  Division  of  CALL  is  the  unit  responsible  for  the 
facilitation  of  data  collection,  processing  of  observations,  and  dissemination  of  lessons 
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learned  via  CALL  products.  The  primary  IT  tool  used  for  this  formulation  of  collection 
plans,  categorizing  of  observations,  and  supplying  of  trend  analysis  is  the  CALL 
Collection  and  Observation  Management  System  (CALLCOMS).  CALLCOMS  supplies 
the  Army  with  an  extremely  effective  process  to  automate  the  collection  and  analysis  of 
lessons  learned.  Additionally,  it  provides  training  support  packages  tailored  to  specific 
units  and  publishes  numerous  periodicals  that  directly  address  the  correction  of  mistakes 
for  unit  commanders  and  the  total  Army.  An  historical  record  of  past  trend  analysis  of 
NTC  units  is  available  to  anyone  with  a  desktop  computer  via  the  CALL  homepage. 
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VI.  FINDINGS 


A.  INTRODUCTION 

This  section  describes  the  research  method  and  sources  of  archival  data  collected 
during  research.  An  in-depth  survey  with  a  large  population  was  beyond  the  scope  of  the 
thesis.  Instead,  an  exploratory  questionnaire  was  sent  to  a  number  of  Naval  Postgraduate 
School  students  who  have  recently  returned  from  the  fleet.  This  resulted  in  an  extremely 
low  number  of  usable  results. 

A  request  was  also  made  to  a  small  number  of  fleet  units.  The  resultant  response 
here,  too,  proved  insufficient  for  analysis.  In  an  effort  to  estimate  the  exposure  of  NLLS 
to  the  fleet,  a  different  approach  was  undertaken.  To  obtain  as  wide  a  range  of  experience 
as  possible,  while  being  cognizant  of  the  extremely  low  return  rate,  individual 
communities  were  specifically  chosen  to  provide  input  for  the  survey.  This  was 
accomplished  by  soliciting  various  members  of  the  targeted  warfare  specialties  until  an 
affirmative  response  was  attained  from  a  member  of  that  community. 

The  questionnaire  was  divided  into  three  sections.  The  first  section  covered  the 
respondent’s  background,  community,  and  experience.  The  second  section  was  aimed  at 
the  Navy  Lessons  Learned  Submission  Process.  Here,  the  questions  were  concerned  with 
the  respondent’s  knowledge  of  the  NLLS  and  frequency  of  individual  or  command 
lessons  learned  submission.  The  final  section  covered  the  retrieval  of  prior  lessons 
learned  as  well  as  the  respondent’s  sense  of  the  relative  effectiveness  of  the  system. 
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It  is  important  to  note  the  survey  was  not  intended  to  be  suggestive  of  a  Navy¬ 
wide  representation.  Instead,  it  was  designed  to  be  a  generalized,  sample-based 
questionnaire  specifically  for  the  purpose  of  this  research.  In  light  of  the  data,  it  is 
recommended  that  a  larger  sample  size  survey  be  undertaken.  All  questions  were  asked 
in  the  past  tense  to  reflect  the  time  the  respondents  were  in  the  fleet  rather  than  stationed 
at  the  Naval  Postgraduate  School. 

B.  SAMPLE  DEMOGRAPHICS 

The  sample  candidates  were  individually  chosen  based  on  a  community  that  they 
represented.  The  pool  of  respondents  consisted  of  two  Aviators,  two  Surface  Warfare 
Officers,  one  Intelligence  officer,  one  Submariner,  one  Medical  Services  Officer,  and  one 
Fleet  Support  Officer. 

The  respondent’s  rank  ranged  from  a  junior  lieutenant  to  a  senior  lieutenant 
commander.  Two  of  the  respondents  had  prior  enlistment  time.  The  positions  held  while 
on  sea  duty  covered  a  wide  range  of  operations  experience  (Squadron  Ops  Officer, 
Combat  Information  Center  Officer,  Assistant  Navigator,  and  Sub  Pilot)  as  well  as  shore 
based  IT  related  positions  (ADP  Officer  and  IT  Project  Officer).  All  but  one  had  served 
aboard  a  ship,  and  those  who  had  served  aboard  ship  had  made  at  least  one  deployment. 
Half  of  the  replies  came  from  personnel  that  had  served  onboard  large  decks  (CVN  or 
LPD).  The  average  number  of  deployments  of  those  who  did  serve  aboard  ship  was  two. 
The  sample  was  composed  of  seven  male  and  one  female  Naval  Officers. 
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C.  RESULTS  AND  DISCUSSION 

1.  The  Navy  Lessons  Learned  Submission  Process 

a.  Have  you  ever  heard  of  the  Navy  Lessons  Learned  Center? 

This  first  question  was  designed  to  get  an  overall  feel  for  the  respondent’s 
knowledge  of  the  NLLS  in  order  to  be  able  to  more  accurately  draw  conclusions  from  the 
rest  of  the  survey.  Of  the  eight,  five,  or  63  percent,  had  heard  of  NLLS.  Two  of  the  “no” 
answers  came  from  aviators,  and  one  from  the  submariner. 


yes  no 


b.  Did  you  know  that  the  Navy  Lessons  Learned  Center  has  a  Web 
site  with  a  database  of  all  the  Lessons  Learned  submitted  in  the 
past  two  years? 


Yes  No 
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This  second  question  was  designed  to  assess  the  respondent’s  knowledge 
of  the  NLLS  system.  Whereas  most  people  had  heard  of  the  system,  only  25  percent  knew 
about  the  Web  site. 


c.  When  your  command  concluded  an  exercise,  were  the  lessons 
learned  collected  and  forwarded  to  your  ISIC? 


i 

Yes  Sometimes  No  Unknown  Local 

The  majority  of  the  respondents,  or  75  percent,  felt  that  lessons  learned 
were  being  collected  either  all  of  the  time  or  at  least  some  of  the  time.  There  was  one 
“no”  answer,  and  that  came  from  the  Medical  Services  Officer. 

Breaking  down  the  “yes”  answers,  three  of  the  six  collected  lessons 
learned  after  every  exercise,  two  of  the  six  after  most  exercises,  and  one  of  the  six 
collected  lessons  learned,  but  these  were  known  to  stay  within  the  command  (Fleet 
Support).  The  sole  “unknown”  response  came  from  the  Intel  Officer. 


□  Yes 

■  Sometimes 

□  No 

□  Unknown 

■  Local 
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d.  During  a  scheduled  exercise  or  evolution,  was  lessons  learned 
collection  given  consideration? 


The  majority  of  respondents  indicated  that  their  commands  gave  lessons 
learned  collection  consideration  during  planning.  The  two  negative  responses  were  from 
the  Intel  Officer  and  the  Medical  Services  Officer. 


e.  Did  you  consider  submitting  a  NLLS  report  a  worthwhile  task? 


Almost  every  respondent  had  a  pessimistic  reaction  toward  the  utility  of 
submitting  lessons  learned.  The  two  positive  responses  came  from  the  Submariner  and 
the  Fleet  Support  Officer. 
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2.  The  Navy  Lessons  Learned  Retrieval  Process 

The  previous  section  revealed  that  almost  everyone  was  either  familiar  with  the 
lessons  learned  system  or  was  a  participant  in  submitting  lessons  learned.  But 
organizational  learning  requires  that  knowledge  gained  by  one  member  or  unit  be 
transferred  to  the  other  members.  This  section  of  the  survey  was  aimed  at  determining  if 
members  were  seeking  out  the  knowledge  that  resided  in  nodes  other  than  their  own. 


a.  Did  your  department  have  access  to  the  SIPRNET ? 


□  Yes 
■  No 


The  most  up-to-date  lessons  learned  information  can  be  accessed  via  the 
SIPRNET.  In  the  event  the  service  member  does  not  have  access  to  the  SIPRNET,  CD- 
ROMs  with  the  same  information  are  distributed  on  a  quarterly  basis. 

This  question  is  getting  at  two  important  issues,  the  ability  to  access  the 
most  current  observations,  and  a  means  for  providing  a  conduit  to  receive  audio,  video,  or 
bulletin  board  information  that  can  be  used  to  enhance  the  recipient’s  understanding  of 
the  message. 

Two  of  the  three  negative  responses  came  from  the  Submariner  and  the 
Medical  Services  Officer.  For  the  Submariner,  the  lack  of  SIPRNET  is  understandable. 
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The  MSO’s  lack  of  SIPRNET  can  most  likely  be  attributed  to  that  community’s  lack  of 
secure  need.  The  final  negative  response  came  from  Aviator  One.  The  ship  he  served  on 
was  non  IT-21  compliant  at  the  time. 

b.  Have  you  visited  the  NLLS  web  or  SIPRNET  site? 


Yes  No 

Only  one  of  the  eight  had  actually  visited  the  NLLS  Web  or  SIPRNET 
site.  Because  this  positive  response  came  from  the  Medical  Services  Officer,  it  was 
unexpected.  In  answering  the  previous  question  the  MSO  cited  a  lack  of  SIPRNET 
access.  When  queried  about  this  response,  the  respondent  indicated  he  had  accessed  the 
NLLS  Internet  Web  site.  The  NLLS  Internet  Web  site  provides  an  overview  of  the  system 
but  does  not  contain  actual  lessons  learned.  It  does  provide  information  on  how  to  access 
them  via  the  SIPRNET  as  well  as  provide  a  means  for  inclusion  on  the  CD-ROM 
distribution  list. 
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c. 


Did  you  have  access  to  either  the  classified  or  unclassified 
Lessons  Learned  CD-ROM? 


□  Yes 
■  No 

□  Unclass 


Sixty-three  percent  responded  that  they  did  not  have  access  to  the  NLLS 
CD-ROM.  There  were  two  affirmative  answers  from  the  Surface  Warfare  Officers,  and 
the  Medical  Services  Officer  proffered  the  single  unclassified  answer. 

The  high  number  of  negative  answers  can  most  likely  be  attributed  either 
to  the  way  the  question  was  asked  or  to  a  lack  of  respondents’  knowledge  of  the 
availability  of  the  CD-ROM.  It  is  certain  most  ships  receive  the  lessons  learned  CD-ROM 
at  a  department  level.  If  the  problem  is  a  lack  of  knowledge  of  the  CDs,  increasing 
awareness  may  help  to  resolve  this. 
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d.  Have  you  searched  the  Lessons  Learned  Data  Base  for  a  Lessons 

Learned? 


Yes  No 

Most  respondents  answered  “no”  to  this  question.  Of  the  three  “yes” 
answers,  two  came  from  the  Surface  Warfare  Officers  and  one  from  the  Medical  Services 
Officer. 

e.  What  type  of  Lessons  Learned  were  you  searching  for  ( Previous 

exercise ,  similar  exercise,  port  visit,  etc.)? 


Port  Prev  IT 
visit  Deply 

This  question  garnered  a  high  level  of  affirmative  responses.  Five  out  of 
the  eight  respondents  reported  accessing  the  database  or,  more  correctly,  a  database.  The 
two  Surface  Warfare  Officers  were  looking  for  port  visit  lessons  learned.  The  two 
Aviators,  who  previously  answered  negatively  to  searching  the  NLLS  database,  explained 
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they  had  searched  or  read  Air  Wing  Lessons  Learned  information.  The  last  response  was 
from  the  Medical  Services  Officer,  and  he  wrote  he  was  searching  for  IT  lessons  learned. 

f.  On  a  scale  of  one  to  five,  where  one  is  of  little  use,  three  is  of 
moderate  use,  and  five  is  of  exceptional  use,  rate  the  usefulness 
of  prior  lessons  learned  as  they  applied  to  you  in  a  exercise  or 
evolution. 


"r  " 
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Aviatori  Aviator2  SWOI  SW02  Intel 


□  Aviatori 

■  Aviator2 

□  SWOI 

□  SW02 

■  Intel  O 


The  aviators  rated  the  usefulness  of  lessons  learned  fairly  high,  with  both 
giving  lessons  learned  research  prior  to  an  exercise  a  four.  Both  of  the  SWOs  stated  that 
they  used  it  for  port  visit  information,  but  one  gave  it  a  three,  and  the  other  responded 
with  a  rating  of  one.  The  last  respondent,  the  Intel  Officer,  gave  it  a  moderate  three. 

g.  On  an  average  of  how  many  times  a  month  did  you  access  the 
lessons  learned  database? 

This  question  revealed  that  all  participants  very  rarely  accessed  the 
database  on  a  regular  basis.  One  respondent  (SWO)  indicated  that  maybe  in  a  good  month 
he  would  access  it  twice,  but  for  the  remainder  the  answer  was  zero  times  or  not 
applicable. 
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h.  When  your  command  prepared  for  an  exercise,  was  the  lessons 

learned  database  consulted  during  planning? 


Yes  No 

This  question  resulted  in  two  affirmative  responses,  one  by  a  SWO  and  the 
second  by  the  Submariner.  However,  in  both  cases,  they  clarified  their  responses  and 
added  that  the  lessons  they  consulted  were  on  their  own  ship’s  database. 

i.  What  type  of  circumstance  would  cause  you  to  reference  the 

lessons  learned  database? 

The  modal  response  to  this  question  was  that  a  service  member  would 
access  the  lessons  learned  database  if  he  or  she  was  going  to  take  part  in  or  undertake  an 
infrequent  evolution.  The  Fleet  Support  Officer,  Submarine  Officer,  and  one  of  the 
Surface  Warfare  Officers  expressed  this  feeling.  Other  common  responses  were  that 
reference  was  made  prior  to  an  exercise,  which  was  stated  by  the  Intelligence  and  Medical 
Officers,  and  if  the  respondent  believed  there  was  corporate  knowledge  residing  in  the 
database,  which  was  expressed  by  the  two  aviators. 
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3. 


NLLS  Effectiveness 


a.  Did  you  consider  reviewing  the  NLLS  database  prior  to  an 
exercise  a  worthwhile  task? 

The  two  respondents  who  answered  “yes”  to  question  eight  of  the  retrieval 
process  both  answered  “yes”  to  this  one.  Of  the  other  six  “no”  or  “not  applicable” 
responses,  four  felt  if  they  had  known  it  was  there,  they  would  have  felt  it  a  worthwhile 
task.  The  two  remaining  answers  were  from  the  aviators  who  remarked  that  reading 
lessons  learned  prior  to  an  exercise  was  very  much  exercise  dependent.  Typically  each 
exercise  tended  to  be  pretty  specific.  After  the  exercise  started,  lessons  learned  were 
almost  always  read,  either  from  previous  missions  or  those  put  out  during  a  squadron  or 
ready  room  meeting. 

b.  Had  referencing  the  Lessons  Learned  database  resulted  in  a 
change  in  the  way  you  would  approach  tactics ,  techniques,  or 
procedures  (TTP)? 


Yes  No  N/A 

The  two  “yes”  answers  in  this  question  came  from  the  aviators,  though 
both  stated  that  they  did  not  refer  to  NLLSDB  Lessons  Learned;  it  was  Air  Wing  or 
Squadron  Lessons  Learned.  Both  stated  that  any  change  that  resulted  was  either  in 
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techniques  or  procedures.  Tactics  generally  were  developed  during  Air  Wing  training,  and 
the  exercises  sought  to  improve  on  them. 

Of  the  three  “no”  answers,  two  came  from  the  Surface  Warfare  officers  with  no 
explanation,  and  one  from  the  Submariner,  who  stated  that  if  he  had  known  it  was  there, 
it  might  have  changed  the  way  he  would  do  something.  The  remaining  three  “not 
applicable”  answers  were  from  the  Fleet  Support,  Intelligence,  and  Medical  Services 
Officers. 


c.  Where  would  you  say  you  received  the  majority  of  your  lessons 

learned  information? 


H  Professional  Journals 
■  Message  Traffic 
□  Meetings 


The  responses  received  from  this  question  can  be  grouped  into  three  main 
themes:  professional  journals,  message  traffic,  and  meetings  with  people  who  shared 
corporate  knowledge.  Three  out  of  the  eight  cited  their  community’s  periodicals 
(Approach  or  Fathom),  which  they  read  on  a  regular  basis.  Two  who  cited  message  traffic 
said  it  included  mishap  reports.  And  the  final  two  cited  either  scheduling  or  planning 
conferences  or  the  informal  network  of  fellow  co-workers  as  the  source. 
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d.  Do  you  have  any  inputs  that  you  feel  would  make  NLLS  a  more 

effective  tool? 

Four  of  the  respondents  felt  NLLS  would  be  a  more  effective  tool  if  the 
awareness  of  the  system  could  be  improved.  This  conclusion  can  be  anticipated  in  part 
because  of  their  lack  of  knowledge  of  the  system  going  into  the  survey. 

One  reply  from  a  Surface  Warfare  Officer  sought  to  make  it  more  user 
friendly.  He  felt  that  writing  each  lesson  learned  took  a  tremendous  of  time,  since  it  had 
to  be  dissected  or  parsed  into  rigid  NLLS  format  (observation,  recommendation,  etc). 

The  remaining  three  provided  no  input  to  improve  the  product. 

D.  ARCHIVAL  DATA  RESEARCH 

1.  Introduction 

As  part  of  this  research,  an  analysis  was  conducted  to  determine  the  feedback 
generated  from  one  of  the  most  strenuous  levels  of  training  encountered  during  a 
battlegroup’s  preparation  for  deployment,  the  Joint  Task  Force  Exercise.  (JTFEX) 

2.  Background 

Following  a  deployment,  ships  usually  enter  into  a  planned  maintenance  and 
upgrade  yard  period.  Upon  completion  of  this  yard  period,  the  ship  will  once  again  rejoin 
the  fleet  returning  to  sea.  These  first  few  at-sea  periods  consist  of  individual  or  unit  level 
training.  Following  this  phase,  the  ship  will  undergo  an  extremely  high  intensity 
integrated,  underway  and  inport  interdployment  workup  cycle  (LDTC)  that  culminates  in  a 
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certification  exercise  called  the  Joint  Task  Force  Exercise.  Throughout  this  exercise  the 
Carrier  Battle  Groups  (CVBG)  and  Amphibious  Ready  Groups  (ARG)  are  presented  with 
a  scenario  that  emphasizes  expeditionary  warfare  in  a  joint  littoral  environment.  These 
exercises,  which  can  involve  many  thousands  of  personnel,  present  increasingly  complex 
scenarios  to  the  participants. 

For  example,  during  JTFEX  99-1,  more  than  24,000  U.S.,  Joint,  and  Allied 
service  members  were  involved  in  exercises  that  took  place  from  the  coast  of  Virginia  to 
the  Islands  of  Puerto  Rico.  In  addition  to  the  Army,  Navy,  Air  Force,  and  Marine  forces, 
elements  from  Great  Britain,  the  Netherlands,  France,  Belgium,  Canada,  Germany,  and 
Bolivia  participated.  The  JTFEX  is  designed  to  provide  realistic  training  to  fully  prepare 
the  ships  and  personnel  for  any  and  all  operations  they  may  encounter  when  deployed. 
The  participants  train  using  equipment  and  systems  that  represent  the  latest  advances  in 
technology.  During  these  exercises  forces  are  tested  on  their  ability  to  deploy  rapidly, 
conduct  joint  operations,  and  refine  tactics,  techniques,  and  procedures. 
(http  ://www.chinfo. navy.mil/)  JTFEX ’s  have  been  called  graduate  level  training  for  the 
battle  groups,  and  successful  completion  is  required  prior  to  being  certified  ready  to 
deploy. 

3.  Methodology 

To  provide  input,  a  search  of  the  Navy  Lessons  Learned  Data  Base  was  conducted 
for  the  last  two  years  of  observations.  This  resulted  in  returns  for  five  Joint  Task  Force 
Exercises.  The  following  three  data  points  were  investigated: 
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•  Total  number  of  lessons  learned  for  the  exercise. 


•  Time  required  to  validate  the  lesson  learned. 

•  Outcome  resulting  from  the  lesson  learned  submission. 

These  outcomes  were  categorized  into  four  classifications.  The  lesson  learned 
submission  could  result  in  a  Tactical  Development  Evaluation  update  (TACMEMO), 
Remedial  Action  Program  (RAP)  working  group  submission,  “disagree”  with  the 
observation,  or  be  delineated  by  either  no  remarks  or  “concur”  with  the  writers’ 
observation.  In  both  of  these  latter  cases,  the  outcome  was  classified  as  “concur.” 

The  significance  of  the  four  categories  is  marked  by  what  further  action  the  lesson 
learned  generates.  A  TACMEMO  update  can  result  in  a  change  to  existing  doctrine, 
tactics,  techniques,  or  procedures  currently  in  place.  A  RAP  item  identifies  a  deficiency  in 
policy,  organization,  training,  education,  or  equipment.  In  both  of  these  cases  the 
knowledge  gained  from  the  lesson  learned  will  be  promulgated  throughout  the 
organization  in  the  form  of  new  publications,  doctrine,  or  equipment. 

In  the  case  of  a  lesson  learned  classified  as  “concur”  or  no  comments  noted,  for 
this  knowledge  to  be  disseminated,  the  organization’s  members  have  to  actively  search 
the  database. 
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4. 


Results 


a.  JTFEX  98-1 


Lessons  TACMEMO  RAP  Concur 
Learned 


The  first  JTFEX  took  place  from  January  12th  to  February  4th  of  1998. 
JTFEX  98-1  consisted  of  the  USS  John  C.  Stennis  Battlegroup  (CVBG)  and  the  USS 
Wasp  Amphibious  Readiness  Group  (ARG).  The  lessons  learned  for  JTFEX  98-1  consist 
of  five  entries.  Validation  for  these  five  entries  took  four  months,  and  the  outcome  for  all 
five  is  classified  as  “concur.” 


b.  JTFEX  98-2 


- 1 - - 1 - - — i  .  i  "i 

Lessons  TAC  RAP  Concur  Disagree 


Learned 

* 
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JTFEX  98-2  was  held  from  April  27th  through  May  13th.  It  consisted  of 
the  USS  Dwight  D.  Eisenhower  Battlegroup  and  the  USS  Saipan  Amphibious  Readiness 
Group.  The  lessons  learned  for  JTFEX  98-2  consisted  of  14  observations.  Validation  for 
these  observations  took  five  months  and  the  outcome  consisted  of  three  TACMEMOS, 
ten  “concur,”  and  one  “disagree.” 


c.  JTFEX  99-1 


Learned 


□  Lessons  Learned 
■TAC 

□  RAP 

□  Concur 


JTFEX  99-1  took  place  from  February  12th  until  March  7th.  Taking  part 
were  the  USS  Theodore  Roosevelt  Battlegroup  and  the  USS  Kearsarge  Amphibious 
Readiness  Group.  The  lessons  learned  for  JTFEX  99-1  consisted  of  29  observations. 
Validation  for  these  observations  took  four  months  and  the  outcome  consisted  of  one 
TACMEMO,  one  RAP,  and  27  “concur.” 


* 
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d. 


JTFEX99-2 


Learned 


□  Lessons  Learned 
■TACMEMO 

□  RAP 

□  Concur 
■  Disagree 


JTFEX  99-2  was  held  from  July  18th  through  July  30th.  Taking  part  in  it 
were  the  USS  John  F.  Kennedy  Battlegroup  and  the  USS  Bataan  Amphibious  Readiness 
Group.  The  lessons  learned  for  JTFEX  99-2  consisted  of  1 1  observations.  Validation  for 
these  observations  took  two  months,  and  the  outcome  consisted  of  ten  “concur”  and  one 
“disagree.” 


e.  JTFEX  00-2 


- 1  i  i  — ■ 

Lessons  TACMEMO  RAP  Concur 
Learned 


JTFEX  00-2  took  place  from  May  10th  until  May  20th.  The  USS  George 
Washington  led  the  carrier  battle  group,  and  the  USS  Saipan  led  the  Amphibious 
Readiness  Group.  The  lessons  learned  for  JTFEX  00-2  consisted  of  27  observations. 
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Validation  for  these  observations  took  seven  months,  and  the  outcome  consisted  of  one 


TACMEMO,  four  RAP  items,  and  22  concurs. 


/.  Overall 


Lesson  TACMEMO  RAP  Disagree  Concur 
Learned 


Overall  for  the  five  Joint  Task  Force  Exercises,  there  were  86  Lessons 
Learned  in  the  database.  From  these  86  observations,  five  TACMEMOSs  and  five  RAP 
items  resulted.  Of  the  remaining  76  observations,  two  were  classified  as  “disagree”  and 
74  “concur.” 
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VII.  ANALYSIS 


A.  INTRODUCTION 

The  goal  of  this  section  is  to  provide  analysis  of  the  Navy  Lessons  Learned 
System  to  determine  if  Organizational  Learning  is  taking  place.  Using  the  theories  of 
Huber,  Argyns  and  Schon,  and  Fiol  and  Lyles  as  a  basis  for  exploration,  we  examine 
whether  there  are  systems  in  place  that  encourage  and  enable  learning.  In  addition, 
emphasis  is  placed  on  examining  the  results  of  the  sample  survey  to  see  if  there  are 
cultural  or  environmental  factors  whose  effects  could  result  in  rendering  even  a  well- 
designed  information  system  ineffective. 

B.  IS  ORGANIZATIONAL  LEARNING  TAKING  PLACE? 

1.  Introduction 

This  analysis  uses  a  framework  adapted  from  a  Department  of  the  Navy  model 
that  presents  the  requirements  for  a  balanced  knowledge  management  approach. 
(Navigating  the  World  of  Knowledge,  2000)  This  model  states  that  balance  is  required  to 
ensure  that  undue  emphasis  is  not  placed  on  any  one  single  aspect  of  management.  It 
contends  that  technology  alone  is  not  sufficient  for  knowledge  transfer.  The  behavior  of 
people  must  be  changed,  and  they  must  be  provided  with  the  tools  to  use  the  technology. 
(Navigating  the  World  of  Knowledge,  2000)  The  framework  comprises  five  separate  but 
equally  important  pieces:  Content,  Process,  Culture,  Learning,  and  Technology.  Using 
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this  as  a  framework  provides  us  with  a  useful  platform  from  which  to  analyze  the 
organizational  learning  models  discussed  earlier  with  the  Navy  Lessons  Learned  System. 


Figure  2.  Organizational  Learning  Model 


2.  Content 

Organizational  Learning  begins  at  Content.  For  the  NLLS  system  this  content 
manifests  itself  in  the  database  of  lessons  learned.  As  we  have  seen,  content  is  made  up  of 
much  more  than  a  flat  file  of  information.  For  lessons  learned  to  be  credible,  the  content 
must  be  timely,  relevant,  and  provide  value  for  the  user. 

For  our  purpose,  timeliness  of  the  information  relates  to  how  long  it  takes  for  the 
information  or  knowledge  gained  by  one  unit  to  be  shared  throughout  the  organization. 


78 


What  is  timely  also  can  depend  on  the  urgency  of  the  situation  or  the  significance  of  the 
lesson  learned.  If  a  ship  is  engaged  in  hostilities,  any  delay  in  sharing  information  may 
result  in  losses.  If  the  ship  is  tied  up  to  the  pier,  the  applicability  of  the  knowledge  may 
not  be  apparent  or  needed  for  many  months. 

For  the  research  done  on  Joint  Task  Force  Exercises,  the  Navy  Lessons  Learned 
System’s  average  time  from  submission  to  publication  was  a  little  under  four  and  a  half 
months.  For  organizations  that  do  not  have  SIPRNET  access  and  rely  on  the  next  CD- 
ROM  distribution,  an  additional  three  months  might  be  necessary.  JTFEXs  are  held 
approximately  every  five  months.  In  this  instance,  getting  the  information  from  the 
previous  battlegroup  experiences  in  time  for  preparations  and  planning  may  not  occur. 

Relevance  and  value  are  closely  related.  Relevance  can  be  thought  of  as  the 
applicability  of  the  information  to  the  individual’s  daily  routine  or  task  carried  out.  Value 
is  the  benefit  the  individual  receives  from  accessing  this  information.  The  value  of  the 
information  can  diminish  with  the  increased  time  required  to  obtain  it. 

A  frequent  comment  heard  when  discussing  lessons  learned  has  to  do  with  the 
relevance  or  applicability  of  the  information  to  the  individual.  The  database  comprises 
approximately  24,000  lessons  learned.  It  is  apparent  from  the  research  done  on  the 
JTFEXs  that  most  lessons  learned  are  very  specific  in  nature,  and  the  applicability  for 
other  individuals  would  appear  to  be  very  low.  There  are  full  text  search  engines  available 
to  look  for  certain  topics  in  the  database,  but  if  the  intention  is  to  get  an  overall 
impression  of  the  exercise  prior  to  undertaking  it,  the  information  is  generally  too  specific 
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to  be  of  use.  The  reason  for  this  becomes  evident  as  one  searches  through  the 
information. 

The  JTFEX  database  comprises  86  observations,  and  the  complete  NLLS  database 
is  approximately  24,000  lessons  learned.  But  most  of  these  observations  consist  of  a 
sample  size  of  one,  which  presents  individuals  and  analysts  with  difficulties  in 
determining  how  valuable  the  knowledge  might  be.  Most  of  these  observations  lack  the 
richness  or  breadth  needed  to  provide  a  wide  increase  in  organizational  knowledge.  On 
the  other  hand,  some  events  provide  a  wealth  of  information  that  is  applicable 
organizational  wide.  A  recent  example  is  the  bombing  of  the  USS  Cole  and  the  lessons 
learned  that  could  be  gained  from  that  ship’s  experiences  refueling  in  a  foreign  port. 

On  a  somewhat  smaller  scale  are  the  lessons  learned  with  every  aircraft  mishap  or 
ship’s  collision.  In  any  of  these  instances  trained  investigators  are  dispatched  to  examine 
events  that  led  up  to  the  incident.  From  their  analysis,  significant  lessons  learned  are 
written  up  and  disseminated  to  the  fleet.  When  the  sample  size  of  one  consists  of 
something  as  rich  as  an  aircraft  or  ship  mishap,  the  relevance  to  every  aircrew  or  Officer 
of  the  Deck  is  apparent.  The  value  is  much  less  apparent  when  searching  through  a  large 
numbers  of  specific  instances  for  one  that  may  be  relevant. 

3.  Technology  and  Process 

In  Huber’s  work  the  role  of  technology  in  organizational  learning  was  explicitly 
specified  as  benefiting  Organizational  Memory.  He  stated  that  the  hard  information  of  a 
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company  was  resident  in  computer  memories  with  the  soft  information  and  the  ability  to 
pass  this  on  residing  in  the  members.  (Huber,  1991) 

With  the  far-reaching  effects  of  the  Internet,  today’s  technology  can  have  a 
profound  effect  on  not  only  Organizational  Memory,  as  Huber  theorized,  but  on  the  other 
three  processes  of  learning,  Knowledge  Acquisition,  Information  Interpretation,  and 
Information  Distribution,  thus  dramatically  increasing  a  member’s  ability  to  pass  this 
knowledge  throughout  the  organization. 

a.  Knowledge  Acquisition 

Knowledge  Acquisition  is  facilitated  by  NLLS  when  available 
technologies  (Web  based  gateways)  are  used  to  scan  the  environment  (NLLSDB)  for 
information  retrieval.  For  the  military  members  using  NLLS,  this  is  a  closed  system,  as 
the  learning  that  occurs  comes  from  within  the  organization. 

Knowledge  Acquisition  also  can  emerge  from  the  revision  of  existing 
doctrine  or  innovations  in  strategy  that  result  from  inputs  to  the  system.  TACMEMO 
publications  and  RAP  item  submissions  are  the  best  examples  of  this. 

Lessons  learned  also  have  shown  themselves  to  be  generators  of  “Best 
Practices”  that  are  considered  to  be  superior  in  approach  and  results.  Usually  Best 
Practices  represent  SME  experiences,  but  they  also  have  been  based  on  inputs  from  the 
fleet  or  research  conducted  in  response  to  an  observation. 
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b.  Information  Distribution 

Information  Distribution  is  the  process  by  which  an  organization  shares  its 
information  among  its  members,  thereby  promoting  knowledge.  For  the  NLLS  system 
this  is  accomplished  in  two  parts.  The  first  is  the  distribution  by  either  the  quarterly 
circulation  of  the  CD-ROM  database  or  the  weekly  updating  of  the  SIPRNET  database. 
The  second  is  the  accessing  of  the  web  site  by  members. 

The  Navy  Lessons  Learned  Organization  information  distribution  process 
takes  the  Field  of  Dreams  approach,  the  “If  you  build  it,  they  will  come”  mindset.  But 
relying  solely  on  this  approach  is  risky.  Based  on  information  provided  by  NWDC, 
during  the  period  of  May  to  October  of  2000  the  total  number  of  lessons  learned  accessed, 
excluding  port  visits  and  RAP  items,  was  27,819.  This  breaks  down  to  approximately 
4,600  hits  a  month.  For  the  research  on  the  Joint  Task  Force  Exercises,  I  accessed  86 
lessons  learned.  In  the  fleet  the  average  number  of  lessons  learned  searched  is  unknown 
but  would  probably  be  closer  to  twenty  per  person.  Twenty  lessons  learned  accessed  per 
person,  per  visit,  divided  by  the  4,600  hits,  would  mean  that  approximately  230  people  a 
month  visited  the  site.  Since  the  majority  of  the  information  contained  in  the  database  is 
only  provided  if  the  user  accesses  the  site,  there  is  a  minimal  amount  of  Information 
Distribution  occurring,  even  though  the  technology  allows  all  members  access. 

c.  Information  Interpretation 

Process  in  this  framework  is  the  means  by  which  the  organization 
captures,  categorizes,  and  presents  information.  As  important  as  capturing  the 
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information  is,  capturing  the  context  is  even  more  so.  Huber  talks  about  this  as 
Information  Interpretation,  or  the  manner  by  which  distributed  information  can  be  given 
one  or  more  commonly  understood  meanings.  (Huber,  1991)  Context  is  unique  at  any 
point  in  time,  and  a  change  in  context  can  result  in  completely  different  interpretations 
between  individuals.  These  interpretations  may  be  subject  to  changing  environment, 
recent  events  that  have  transpired,  or  even  interactions  between  crew  members  on  a  ship. 
A  knowledge  base  of  lessons  learned  is  particularly  sensitive  to  the  difference  in  context 
that  exists  between  the  sender  of  the  message  and  the  receiver.  Using  the  Joint  Task  Force 
Exercises  as  an  example,  a  particular  lesson  learned  has  behind  it  a  host  of  conditions  that 
may  or  may  not  be  applicable  or  apparent  to  the  receiver.  Everything  from  different 
equipment  configurations  onboard  ship  to  battlegroup  composition  challenges  the 
relevancy  and  comprehension  of  the  intended  meaning  to  another  individual. 

Compounding  this  difficulty,  the  lessons  learned  layout  consists  of  a  four- 
paragraph  text  only  format.  These  paragraphs  are  broken  down  into  an  Observation, 
Discussion,  Lesson  Learned,  and  Comments  section.  To  give  the  intended  meaning  a 
greater  chance  of  being  understood,  Information  Technology  can  be  used  as  a  way  to 
enrich  the  message.  Providing  an  opportunity  to  include  drawings,  diagrams,  or 
documentation  in  a  PDF  format  may  make  the  information  more  understandable.  Anyone 
who  has  tried  to  program  a  VCR  understands  the  benefits  of  diagrams  that  are  used  to 
supplement  the  instructions. 

To  increase  the  breadth  of  lesson  learned  applicability.  Information 
Technology  can  also  be  used  to  cluster  or  bring  together  data  and  information  that  is 
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similar  or  related.  Clustering  -  for  example,  on  a  keyword  search  -  would  allow  the  user 
the  ability  to  compare  and  contrast  information  from  a  variety  of  sources  around  a  subject 
or  topic  area.  Even  if  the  lesson  learned  is  not  applicable,  or  the  search  returns  without  a 
match,  there  may  be  valued  gained. 

4.  Learning 

a.  Individual  vs.  Organizational 

To  assess  NLLS  effectiveness  on  learning,  individual  learning  must  be 
separated  from  organizational  learning.  Individual  learning  involves  the  detection  and 
correction  of  errors.  It  can  consist  of  single  loop  or  double  loop  learning.  In  single  loop 
learning,  members  of  the  organization  respond  to  environmental  changes  by  detecting  and 
correcting  errors  in  ways  that  allow  the  organization  to  continue  with  its  present  policies. 
In  double  loop  learning,  the  error  is  detected  and  corrected  in  ways  that  involve  the 
modification  of  the  organization’s  underlying  norms,  policies,  and  objectives.  (Argyris 
and  Schon,  1978)  Learning  at  an  individual  level,  however,  remains  within  the  party  or 
unit  that  created  it. 

Organizational  learning  occurs  when  members  of  the  organization  share 
their  memories,  associations,  or  experiences.  Sharing  can  be  done  by  traditional  methods, 
such  as  meetings  and  memorandums,  or  it  can  be  done  by  computer-generated  means 
such  as  e-mail,  electronic  bulletin  board,  or  file  downloads.  This  organizational  learning 
is  influenced  by  many  factors,  such  as  structure,  strategy,  environment,  technology,  and 
culture.  (Fiol  and  Lyles,  1985)  Organizational  learning  occurs  with  the  acquisition  and 
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transfer  of  knowledge,  and  it  reveals  itself  when  the  organization  modifies  its  behavior  to 
reflect  this  new  knowledge.  (Garvin,  1993) 

Although  learning  remains  essentially  the  same  in  the  individual  case  as  in 
the  organizational  case,  the  learning  process  is  fundamentally  different  at  the 
organizational  level.  (Kim,  1993)  When  we  talk  about  organizational  learning,  we  are 
concerned  with  conveying  knowledge  from  one  person  to  the  next. 

b.  NLLS  Effect  on  Organizational  Learning 
To  evaluate  NLLS  effect  on  organizational  learning,  one  has  to  look  at  the 
individual’s  usage  of  the  system.  Usage  in  this  instance  is  defined  by  an  individual 
submitting  a  lesson  learned  to  distribute  newfound  knowledge,  or  by  an  individual 
accessing  the  information  to  increase  the  breadth  of  the  organization’s  knowledge  base.  It 
is  not  possible  to  accurately  evaluate  the  number  of  times  that  CD-ROMs  are  used  to 
search  the  database.  Instead,  this  opinion  is  based  on  the  research  data  and  information 
provided  by  the  Web  site. 

Overall,  the  sample  survey  and  Web  counter  information  indicate  a  very 
low  rate  of  accessing  the  NLLS  database  by  military  members.  Looking  at  inputs  to  the 
system,  it’s  important  to  remember  that  the  database  contains  only  those  lessons  learned 
forwarded  by  the  individual’s  Immediate  Superior  In  Command  (ISIC),  not  the  total 
number  of  actual  submissions  that  may  have  been  received  after  an  exercise.  That  said, 
for  the  five  Joint  Task  Force  exercises,  which  can  involve  up  to  20,000  participants  each, 
the  average  number  of  lessons  learned  generated  is  1 7  per  exercise. 
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For  organizational  learning  to  occur  through  the  NLLS  system,  active 
participation  by  individual  members  is  required.  The  NLLS  system  relies  on  a  passive 
collection  method  and  an  active  distribution  method.  Inputs  are  generated  from 
individuals  in  the  fleet.  Once  these  observations  are  validated,  CD-ROMs  are  distributed 
and  the  NLLSDB  is  updated.  To  complete  the  dissemination  of  knowledge,  members 
must  actively  pull  the  information  from  the  database. 

Currently,  the  Navy  Lessons  Learned  System  provides  an  extremely 
efficient  way  to  promote  organizational  learning  with  the  automation  of  submission  and 
validation  of  lessons  learned.  However,  the  effectiveness  of  organizational  learning 
suffers  from  the  poor  response  rate  or  participation  rate  by  members  of  all  communities. 
To  demonstrate  organizational  learning,  “that  which  is  stored  in  organizational  memory 
must  be  brought  forth  from  memory”.  (Huber,  1991)  This  poor  response  rate  inhibits  the 
communication  of  new  approaches  or  success  stories  gained  from  within  the  organization. 
Without  greater  involvement,  formalizing  the  process  of  transferring  best  practices  and 
lessons  learned  is  not  going  to  result  in  new  discoveries  or  solutions  being  leveraged  off 
existing  ones. 


5.  Culture 

Lessons  learned  collection  has  become  synonymous  with  exercise  completion.  All 
too  often  it  is  given  perfunctory  thought,  and  the  only  reason  inputs  are  generated  is  that 
they  are  mandated  for  after-action  reports  or  required  from  the  chain  of  command.  More 
significant,  even  this  obligatory  emphasis  placed  on  lessons  learned  collection  is  rarely 


86 


placed  on  lessons  learned  research  prior  to  an  exercise.  The  reasons  can  vary,  but  the 
research  is  often  seen  as  an  inefficient  use  of  time  by  personnel  undertaking  an 
assignment  or  planning  an  operation.  Frequent  comments  echoing  this  came  from 
members  who  stated  that  they  experienced  a  lack  of  benefit  from  the  effort  expended.  For 
the  NLLS  system  the  greatest  hindrance  was  in  the  lack  of  exposure  most  people  had  to 
the  database.  Still,  it  is  not  certain  that  with  greater  exposure  the  response  rate  would 
have  fared  much  better.  The  small  number  of  lessons  learned  generated  from  the  five 
Joint  Task  Force  Exercises  indicates  there  is  a  fundamental  culture  present  in  the  Navy 
that  reduces  lessons  learned  submission  and  hampers  NLLS  ability  to  promote 
Organizational  Learning.  This  culture  may  be  based  in  part  on  the  mindset  seen  in 
Knowing  Organizations,  an  attitude  that  hinders  the  learning  organization’s  culture  of 
“openness  to  experience,  encouragement  of  risk  taking,  and  willingness  to  acknowledge 
failure.”  (McGill  and  Slocum,  1994) 

The  Knowing  Organization  is  a  model  advanced  by  M.  E.  McGill  and  J.  W. 
Slocum  that  can  be  seen  in  operation  in  virtually  every  franchise  operation  in  the  United 
States  today.  In  a  Knowing  Organization,  there  is  a  belief  that  there  is  only  one  best  way 
to  complete  a  task,  manage  an  employee,  or  structure  the  organization. 

Undoubtedly  the  most  famous  and  visible  of  all  knowing  organizations  is 
McDonald’s.  Its  13,000  plus  stores  exemplify  the  best  of  what  a  knowing 
organization  has  to  offer:  efficiency,  predictability,  and  control  in 
production  and  customer  services.  ”  (McGill  and  Slocum,  1994) 
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The  best  way  to  fry  a  hamburger,  or  how  long  the  french  fries  should  be  cooked  in 
oil,  has  been  discovered  and  is  committed  to  the  organizational  memory.  A  Knowing 
Organization  focuses  on  “standardized  policies,  procedures,  rules,  and  regulations.” 
(McGill  and  Slocum,  1994;  Hannah,  1993)  Knowing  Organizations  take  their  strength 
from  the  high  levels  of  conformity  and  standard  operating  procedures  that  form  a  very 
effective  organization  as  long  as  the  environment  remains  relatively  stable. 

In  times  of  change,  the  Knowing  Organization  may  make  “incremental  changes  to 
existing  processes  but  these  are  really  only  tweakings  and  fine  tunings  of  what  the 
organization  already  does.”  (McGill  and  Slocum,  1993;  Hannah,  1993)  In  a  Knowing 
Organization,  managers  develop  practices  over  time  that  protect  the  organization  from 
pressures  and  outside  influences,  whatever  their  merit.  “Rituals,  stories,  jargon,  and 
physical  settings  are  all  instruments  that  can  hinder  their  ability  to  detect  and  develop  the 
mechanisms  that  learning  requires.”  It  is  because  of  this  that  McGill  and  Slocum  believe 
Knowing  Organizations  are  “learning  disadvantaged.”  (McGill  and  Slocum,  1993; 
Hannah,  1993) 

Lessons  learned  are  often  not  written  because  we  use  the  established  doctrine  and 
standard  procedures.  Doctrine  says  how  to  assault  a  beach,  so  we  plan  and  execute 
according  to  doctrine.  If  a  lesson  learned  is  written,  it  tends  to  deal  with  communication 
issues,  circuit  management,  or  even  the  benefits  of  e-mail,  something  that  may  have  great 
applicability  to  the  ship  involved  at  the  time,  but  marginal  utility  to  the  next  battlegroup. 

Small  incremental  changes  to  existing  process  mark  the  sign  of  a  Knowing 
Organization.  The  lessons  learned  resulting  from  Joint  Task  Force  Exercises  should  be 
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initiatives  on  the  scale  of,  “How  many  strike  aircraft  do  we  need  to  accomplish  this 
mission”  Instead,  they  point  out  that  a  ship  may  need  a  reservist  to  help  sort  incoming 
messages. 
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VIII.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

The  Navy’s  Lessons  Learned  Program  has  undergone  significant  and  noteworthy 
changes  since  its  establishment  in  1991.  Today,  the  organization  fully  meets  its  objective 
of  providing  the  Navy  with  a  “low-cost  database  system  to  collate,  evaluate,  and 
disseminate  Navy  specific  lessons  learned.”  (OPNAVINST  3500.37B) 

Though  it  meets  its  purpose,  the  author  believes  there  is  still  room  for 
improvement.  These  improvements  can  be  characterized  by  either  approach  or  processes, 
with  specific  suggestions  covered  in  the  Recommendations  section. 

1.  Approach 

As  technologies  and  operations  evolve,  there  is  little  disagreement  with  the 
increasingly  important  need  to  capture,  record,  and  disseminate  lessons  learned.  However, 
the  goal  should  not  be  to  obtain  as  many  lessons  learned  as  possible.  The  volume  of  the 
database  is  not  nearly  as  important  as  the  quality  of  the  information.  The  Navy  Lesson 
Learned  System’s  focus  should  be  enabling  Knowledge  Creation,  not  Knowledge 
Recording.  The  ultimate  goal  should  be  establishing  a  Knowledge  Management  System 
that  oversees  the  creating,  securing,  capturing,  coordinating,  combining,  retrieving,  and 
distribution  of  knowledge.  (Liebowitz,  2000) 

NLLS  and  Doctrine  Department  are  in  a  unique  position  to  manage  the 
knowledge  environment  at  an  organizational  level.  Learning  lessons  once  is  an  underlying 
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aim;  providing  the  right  information  to  the  right  person  at  the  right  time  is  the  goal.  Many 
organizations,  the  Navy  included,  are  drowning  in  information  and  starving  for 
knowledge.  Targeted  lessons  collection,  specific  knowledge  creation,  and  active 
knowledge  dissemination  should  be  the  objective  of  the  NLLS  system  as  the  Navy  enters 
the  21st  century. 

2.  Process 

The  Navy  Lessons  Learned  System  is  implementing  a  Knowledge  Attic  approach. 
Lessons  learned  are  collected  by  passive  means,  validated  by  active  means,  but  stored  and 
disseminated  by  passive  means.  The  organization  does  not  play  an  active  enough  role  in 
determining  and  targeting  what  it  feels  are  the  areas  the  military  most  needs  to  improve. 
Additionally,  appropriate  lessons  learned  are  not  sent  to  the  individuals  in  the 
organization  who  would  find  these  lessons  to  be  of  the  greatest  value.  Critical  elements 
for  advancing  the  organizational  intelligence  are  identifying  individual  learning,  passing 
it  on,  and  creating  organizational  learning. 

Background  research  indicates  that  at  almost  every  level  in  the  organization, 
lessons  learned  are  being  collected  and  disseminated,  but  to  varying  degrees  and  on 
various  levels.  Whether  it  is  the  trouble-shooting  procedures  developed  in  the  avionics 
shop,  or  an  end-of-cruise  report  prepared  by  a  fighter  squadron,  there  are  varying  degrees 
of  utility  these  lessons  have  to  other  members  in  the  military.  Information  Technology 
has  reduced  the  delay  previously  encountered  in  collecting  and  submitting  lessons  learned 
but  has  resulted  in  stove-piping  of  this  information  among  various  units  or  staffs  within 
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the  organization.  Increased  emphasis  on  networking  should  be  used  to  bring  together 
these  multiple  bodies  of  explicit  knowledge  that  exist.  By  streamlining  requirements  to 
validate  lessons  learned  and  in  some  cases  dropping  them  all  together,  NLLS  can 
accelerate  the  process  of  turning  individual  learning  into  organizational  learning. 
Observations  concerning  tactics,  techniques,  or  procedures  will  still  require  formalized 
methodology  for  validation,  but  there  are  various  other  lessons  learned  that  do  not. 
Knowledge  enablers  consisting  of  Web-based  turnover  pages  or  bulletin  board  discussion 
areas  for  Operations  Officers  would  foster  organizational  learning  and  allow  incoming 
units  a  chance  to  more  quickly  assume  their  operational  mode. 

B.  RECOMMENDATIONS 

1.  Target  the  User 

NLLS’s  most  important  step  should  be  to  undertake  a  more  through  survey  to 
accurately  assess  who  the  customer  is.  The  sample  questionnaire  identified  a  lack  of 
exposure  to  the  system  by  most  of  the  respondents.  By  more  precisely  identifying  the 
customer,  NLLS  can  focus  awareness  of  the  system  to  that  population  and  work  to  ensure 
the  process  meets  their  needs. 

2.  Pull  the  Relevant  Information 

The  Navy  Lessons  Learned  System  relies  on  a  passive  collection  method.  The 
consequence  of  this  is  inputs  of  large  volumes  of  data,  from  whatever  source  is  available, 
without  thought  or  consideration  to  defining  an  overall  collection  plan.  Each  observation 
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requires  investigation  by  a  Subject  Matter  Expert  regardless  of  the  relevant  overall  benefit 
to  the  organization.  The  end  result  is  a  wide-ranging  database  of  discrete  facts  that  took 
many  man-hours  to  complete  and  is  difficult  to  analyze.  Passive  collection  does  have  a 
place  in  NLLS  as  an  enabler  of  input  from  all  levels  of  the  organization.  However, 
targeting  the  focus  of  collection  would  provide  greater  benefit  to  the  organization  as  a 
whole. 

Active  collection  entails  a  coordinated  effort  to  try  to  identify  solutions  to  existing 
problems  or  issues  identified  by  a  sponsor.  This  issue  can  result  from  training  exercises  or 
from  shortcomings  identified  in  actual  operations.  Other  sources  could  be  analysts, 
instructors,  doctrine  writers,  or  commanders  in  the  field.  (Tulak,  1999)  The  environment 
should  be  scanned  for  emerging  trends  or  issues  as  well  as  ongoing  operations  examined 
for  possible  future  ramifications.  The  Early  Bird  can  provide  a  good  starting  point  for 
topics  currently  rising  in  prominence.  (Tulak,  1999) 

Active  collection  options  can  range  from  training  evaluators  to  act  as  observers  to 
identifying  and  targeting  certain  learning  objectives.  Warfare  specialists  or  the  Executive 
Steering  Committee  can  identify  these  learning  objectives,  and  they  may  change  over 
time.  Possible  examples  in  the  aviation  world  would  be  to  track  Blue  on  Blue  incidents  or 
target  acquisitions  rates  during  JTFEXs.  In  the  case  of  target  acquisition,  metrics  would 
be  developed  to  identify  reasons  the  aircrew  failed  to  acquire  a  target.  Data  points  could 
include  weather,  equipment  failure,  or  training.  From  this  information,  trends  would  be 
readily  identifiable  that  may  help  to  improve  target  acquisition  or  identify  shortcomings 
in  training  syllabi.  Measured  performance  almost  always  improves. 
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Cost  is  always  an  issue,  but  specific  exercises  could  be  targeted  for  trained 
observers  based  on  their  rising  importance,  emerging  trends,  or  high  visibility.  JTFEX  00- 
1  provides  us  with  an  example. 

Because  of  the  political  situation  in  Puerto  Rico,  the  USS  Dwight  D.  Eisenhower 
Battlegroup  was  unable  to  conduct  live  naval  surface  fire  support  (NSFS)  training,  live 
fire  air  to  ground  training,  or  combined  arms  training  on  the  Island  of  Vieques.  Efforts 
were  made  to  reschedule,  cancel,  or  move  portions  of  the  training  to  alternate  countries 
while  on  deployment.  Lessons  learned  from  this  exercise  would  have  been  particularly 
valuable  for  follow-on  battlegroups  in  the  event  they  encountered  the  same 
circumstances.  As  important  as  collecting  Eisenhower’s  lessons  learned  would  be 
analyzing  her  trends  on  deployment  due  to  missed  critical  training  during  workups. 

Recent  decisions  by  the  governor  of  Puerto  Rico  have  resulted  in  the  halt  of 
training  on  the  Island  of  Vieques  for  the  USS  Enterprise  Battlegroup  and  the  USS 
Kearsarge  Amphibious  Ready  Group.  With  the  Enterprise  scheduled  to  deploy  in  late 
April,  Air  Wing  Commanders  and  ship  Commanding  Officers  may  be  scrambling  to 
locate  alternate  training  sites  to  ensure  the  readiness  of  their  people.  Unfortunately,  they 
will  not  be  able  to  gain  benefit  from  the  Eisenhower’s  experiences.' Because  of  what  some 
people  believe  was  either  low  submission  rates  or  the  poor  quality  of  lessons  learned 
generated  from  JTFEX  00-1,  the  CINCLANTFLT  Fleet  Management  Site  has  yet  to 
receive  this  information  to  include  in  the  database.  Trained  observers  would  have  helped 
in  this  case,  and  the  lessons  learned  could  have  been  used  to  provide  assessment  for 
upcoming  policy  decisions  on  the  future  of  Vieques. 
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3. 


Push  Information  to  the  Users 


A  common  suggestion  to  improve  NLLS  was  to  increase  awareness  of  the  system. 
One  method  of  accomplishing  this  would  be  to  publish  a  quarterly  newsletter  or  bulletin. 
Success  stories  and  best  practices  could  be  solicited  from  fleet  units,  and  articles  that  are 
published  should  gamer  the  ship  points  in  the  yearly  Battle  “E”  competition. 

An  alternative  method  would  be  to  provide  a  Top  Ten  list  with  a  short  description 
of  the  observation  for  inclusion  into  already  established  publications  like  Fathom  or 
Approach.  The  article  would  let  the  reader  know  where  to  go  to  get  further  information. 
For  ships  scheduled  to  undergo  JTFEX,  or  getting  ready  to  go  on  deployment,  briefing 
packages  of  recent  trends  noted  could  be  provided.  Inputs  for  these  trends  could  be 
provided  by  the  various  SMEs  who  see  the  lessons  learned  on  a  daily  basis  and  targeted 
by  class  of  ship  or  area  of  deployment. 

Finally,  unclassified  lessons  learned  should  be  made  available  over  the  Internet  to 
those  users  who  have  a  .mil  extension.  A  link  could  be  posted  on  the  Navy  homepage 
along  with  information  about  the  site.  Each  of  these  suggestions  may  provide  incentives 
to  use  NLLS  as  well  as  promote  the  system  to  users  throughout  the  fleet. 

4.  Categorize  by  Community 

One  difficulty  noted  while  researching  the  NLLSDB  arose  from  the  way  in  which 
the  information  was  organized.  The  data  is  currently  grouped  under  the  operations  name. 
Noble  Shirley,  Juniper  Stallion,  or  Fortress  Raptor  are  a  few  examples.  Unless  the  user  is 
familiar  with  the  exercise,  extensive  time  may  be  spent  searching  for  comparable 
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situations.  There  is  some  benefit  to  this  structure;  however,  an  alternate  framework  for 
arranging  or  categorizing  the  information  and  knowledge  should  be  used  that  enables 
people  to  find  and  use  it  more  effectively. 

A  simple  sort  function  that  allows  the  user  to  group  observations  by  topic  or 
community  would  be  helpful.  Instead  of  being  grouped  by  exercise  name,  they  could  be 
grouped  under  Missions  with  subheadings  of  Humanitarian  Assistance,  Noncombatant 
Evacuation,  or  Foreign  Disaster  Relief.  Operations  could  have  subheadings  of  Civil, 
Interagency,  Multinational,  or  Combat.  Further  headings  could  include  sections  based  on 
Communications,  Command  and  Control,  or  Intelligence.  Organizing  the  information  by 
platform  or  community  is  one  way  to  facilitate  knowledge  flow. 

5.  Networking  and  Resourcing 

The  NLLS  Organization  with  the  help  of  the  Fleet  Management  Sites  could 
provide  a  critical  role  as  system  intermediaries,  responsible  for  connecting  people  to  the 
information  they  require.  The  various  Fleet  Management  Sites  already  have  a  strong 
familiarity  with  how  to  locate  information  or  expertise  in  the  organization  quickly  and 
efficiently.  Using  this  experience  a  Yellow  Page  Directory,  mapping  knowledge  areas  to 
experts  within  the  organization,  could  be  provided. 

C.  SUGGESTIONS  FOR  FURTHER  STUDY 

This  exploratory  study  has  only  begun  to  uncover  the  growing  role  and  effect  that 
Information  Technology  will  have  on  Organizational  Learning.  The  explosion  of  the 
Internet,  with  all  of  its  benefits  and  all  of  its  drawbacks,  is  in  its  infancy  in  the  military 
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and  will  continue  to  amaze  and  frustrate  the  men  and  women  who  serve  on  board  ships  in 
the  near  future. 

Further  areas  of  study  in  Organizational  Learning  would  include  examining  the 
possibility  of  Web-based  turnover  pages  that  would  allow  the  oncoming  aircraft  carrier, 
as  it  transits  the  Atlantic  or  Pacific  Ocean,  to  take  part  in  the  strike  planning  or  shipboard 
operations  that  will  be  expected  of  it  once  it  is  on  station.  Examining  the  role  and 
implications  that  communities  of  practice  “chat  rooms”  will  have  on  breaking  down 
traditional  boundaries  and  allowing  the  free  flow  of  information  between  the  organization 
and  the  environment  as  they  are  set  up  between  ships  or  even  battlegroups  is  another. 
Along  with  this,  a  question  must  be  answered:  Is  lateral  communication  a  threat  to  good 
order  and  discipline,  blurring  the  lines  between  traditional  structures  and  the  chain  of 
command  that  is  in  place? 

Other  questions  include  what  sort  of  expert  or  decisions  support  systems  the 
Naval  Officer  of  tomorrow  will  require  to  deal  with  the  data  deluge  of  information  that  is 
surely  going  to  take  place.  Lastly,  as  our  reliance  on  Information  Technology  grows,  will 
the  bandwidth  that  is  available  on  board  ship  keep  pace,  or  will  it  place  limitations 
between  ships  in  the  battlegroup,  creating  a  digital  Darwinism  between  the  haves  and  the 
have-nots,  leaving  the  smaller  ships  out  of  the  technological  loop  of  information? 


98 


APPENDIX  A.  SURVEY  INSTRUMENT 


Navy  Lessons  Learned  System  (NLLS)  Questionnaire 

[Thank  you  for  agreeing  to  take  time  to  answers  some  questions.  I  am  examining  the  Navy  Lessons  Learned 
System  and  its  impact  on  organizational  learning.  I  am  conducting  this  research  for  my  Master  Thesis  at 
the  Naval  Postgraduate  School.  This  questionnaire  will  address  three  primary  topics  concerning  the  Navy 
Lessons  Learned  system:  the  submission  process,  the  lessons  learned  retrieval  process,  and  the  effect  the 
NLLS  system  had  on  the  way  you  planned  or  approached  a  scheduled  training  exercise.  The  information  I 
collect  will  be  strictly  confidential  and  your  name  will  not  be  used  in  any  way.  I  appreciate  your  taking  the 
time  to  fill  this  out  and  would  welcome  any  additional  comments  you  have  on  the  system] 


Background 

1 .  What  is  your  rank  and  designator? 

2.  What  positions  have  you  held  while  on  sea  duty? 

3.  What  platforms  did  you  serve  on? 

4.  Did  you  make  any  deployments  while  onboard,  if  so  how  many? 


Lessons  Learned  Submission 

1 .  Have  you  ever  heard  of  the  Navy  Lessons  Learned  Center? 

2.  Did  you  know  that  the  Navy  Lessons  Learned  Center  has  a  Web  site  with  an  active 
database  of  all  the  lessons  learned  submitted  in  the  last  two  years? 

3.  When  your  command  concluded  an  exercise  were  the  lessons  learned  collected  and 
forwarded  to  your  ISIC? 

4.  During  a  scheduled  exercise  or  evolution,  was  lessons  learned  collection  given 
consideration? 

5.  Did  you  consider  submitting  a  NLLS  report  a  worthwhile  task? 
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Lessons  Learned  Retrieval 


1 .  Did  your  department  have  access  to  the  SIPRNET? 

2.  Have  you  visited  the  NLLS  Web  or  SIPRNET  site? 

3.  Did  you  have  access  to  either  the  classified  or  unclassified  lessons  learned  CD-ROM? 

4.  Have  you  searched  the  database  for  a  lesson  learned? 

5.  What  type  of  lesson  learned  were  you  searching  for? 

6.  On  a  scale  of  one  to  five,  where  one  is  of  little  use,  three  is  of  moderate  use,  and  five  is 
of  exceptional  use,  rate  the  usefulness  of  prior  lessons  learned  as  it  applied  to  you  in  an 
exercise  or  evolution. 

7.  On  average  how  many  times  a  month  did  you  access  the  lessons  learned  database? 

8.  When  your  command  prepared  for  an  exercise,  was  the  lessons  learned  database 
consulted  during  planning? 

9.  Did  you  consider  reviewing  the  NLLS  database  prior  to  an  exercise  a  worthwhile  task, 
if  not  why? 

1 0.  What  type  of  circumstance  would  cause  you  to  reference  the  lessons  learned 
database? 

1 1 .  Had  referencing  the  NLLS  database  resulted  in  a  change  in  the  way  you  would 
approach  tactics,  techniques,  or  procedures  (TTP)? 


12.  Where  would  you  say  you  receive  the  majority  of  lesson  learned  information? 


13.  Do  you  have  any  inputs  that  you  feel  would  make  the  NLLS  a  more  effective  tool? 
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